IO scheduler based IO Controller V2
Subsystems (11)
Quoted 13 lines · Vivek Goyal
Quoted 11 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is the V2 of the IO controller patches generated on top of 2.6.30-rc4. First version of the patches was posted here.
Hi Vivek,
I did some simple test for V2, and triggered an kernel panic. The following script can reproduce this bug. It seems that the cgroup is already removed, but IO Controller still try to access into it.
Quoted 9 lines
Hi Gui,
Thanks for the report. I use
cgroup_path()for debugging. I guess thatcgroup_path()was passed null cgrp pointer that's why it crashed.If yes, then it is strange though. I call
cgroup_path()only after grabbing a refenrece to css object. (I am assuming that if I have a valid reference to css object thencss->cgrpcan't be null).
Yes, css->cgrp shouldn't be NULL.. I doubt we hit a bug in cgroup here. The code dealing with css refcnt and cgroup rmdir has changed quite a lot, and is much more complex than it was.
Quoted 2 lines
Anyway, can you please try out following patch and see if it fixes your crash.
...
Quoted 3 lines
BTW, I tried following equivalent script and I can't see the crash on my system. Are you able to hit it regularly?
I modified the script like this:
======================
#!/bin/sh
echo 1 > /proc/sys/vm/drop_caches
mkdir /cgroup 2> /dev/null
mount -t cgroup -o io,blkio io /cgroup
mkdir /cgroup/test1
mkdir /cgroup/test2
echo 100 > /cgroup/test1/io.weight
echo 500 > /cgroup/test2/io.weight
dd if=/dev/zero bs=4096 count=128000 of=500M.1 &
pid1=$!
echo $pid1 > /cgroup/test1/tasks
dd if=/dev/zero bs=4096 count=128000 of=500M.2 &
pid2=$!
echo $pid2 > /cgroup/test2/tasks
sleep 5
kill -9 $pid1
kill -9 $pid2
for ((;count != 2;))
{
rmdir /cgroup/test1 > /dev/null 2>&1
if [ $? -eq 0 ]; then
count=$(( $count + 1 ))
fi
rmdir /cgroup/test2 > /dev/null 2>&1
if [ $? -eq 0 ]; then
count=$(( $count + 1 ))
fi
}umount /cgroup
rmdir /cgroup
======================
I ran this script and got lockdep BUG. Full log and my config are attached.
Actually this can be triggered with the following steps on my box:
# mount -t cgroup -o blkio,io xxx /mnt # mkdir /mnt/0 # echo $$ > /mnt/0/tasks # echo 3 > /proc/sys/vm/drop_cache # echo $$ > /mnt/tasks # rmdir /mnt/0
And when I ran the script for the second time, my box was freezed and I had to reset it.
Quoted 8 lines
Instead of killing the tasks I also tried moving the tasks into root cgroup and then deleting test1 and test2 groups, that also did not produce any crash. (Hit a different bug though after 5-6 attempts :-)
As I mentioned in the patchset, currently we do have issues with group refcounting and cgroup/group going away. Hopefully in next version they all should be fixed up. But still, it is nice to hear back...
#
# Automatically generated make config: don't edit # Linux kernel version: 2.6.30-rc4
# Thu May 7 09:11:29 2009
#
# CONFIG_64BIT is not setCONFIG_X86_32=y
# CONFIG_X86_64 is not setCONFIG_X86=y CONFIG_ARCH_DEFCONFIG="arch/x86/configs/i386_defconfig" CONFIG_GENERIC_TIME=yCONFIG_GENERIC_CMOS_UPDATE=y CONFIG_CLOCKSOURCE_WATCHDOG=y CONFIG_GENERIC_CLOCKEVENTS=y CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y CONFIG_LOCKDEP_SUPPORT=y CONFIG_STACKTRACE_SUPPORT=y CONFIG_HAVE_LATENCYTOP_SUPPORT=y CONFIG_FAST_CMPXCHG_LOCAL=yCONFIG_MMU=yCONFIG_ZONE_DMA=yCONFIG_GENERIC_ISA_DMA=yCONFIG_GENERIC_IOMAP=yCONFIG_GENERIC_BUG=yCONFIG_GENERIC_HWEIGHT=y CONFIG_ARCH_MAY_HAVE_PC_FDC=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set CONFIG_RWSEM_XCHGADD_ALGORITHM=y CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y CONFIG_GENERIC_CALIBRATE_DELAY=y
# CONFIG_GENERIC_TIME_VSYSCALL is not set CONFIG_ARCH_HAS_CPU_RELAX=y CONFIG_ARCH_HAS_DEFAULT_IDLE=y CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y CONFIG_HAVE_SETUP_PER_CPU_AREA=y CONFIG_HAVE_DYNAMIC_PER_CPU_AREA=y
# CONFIG_HAVE_CPUMASK_OF_CPU_MAP is not set CONFIG_ARCH_HIBERNATION_POSSIBLE=y CONFIG_ARCH_SUSPEND_POSSIBLE=y
# CONFIG_ZONE_DMA32 is not set CONFIG_ARCH_POPULATES_NODE_MAP=y
# CONFIG_AUDIT_ARCH is not set CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y CONFIG_GENERIC_HARDIRQS=y CONFIG_GENERIC_HARDIRQS_NO__DO_IRQ=y CONFIG_GENERIC_IRQ_PROBE=y CONFIG_GENERIC_PENDING_IRQ=y CONFIG_USE_GENERIC_SMP_HELPERS=y CONFIG_X86_32_SMP=yCONFIG_X86_HT=yCONFIG_X86_TRAMPOLINE=yCONFIG_X86_32_LAZY_GS=yCONFIG_KTIME_SCALAR=y CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
#
# General setup
#CONFIG_EXPERIMENTAL=yCONFIG_LOCK_KERNEL=yCONFIG_INIT_ENV_ARG_LIMIT=32CONFIG_LOCALVERSION=""
# CONFIG_LOCALVERSION_AUTO is not setCONFIG_HAVE_KERNEL_GZIP=yCONFIG_HAVE_KERNEL_BZIP2=yCONFIG_HAVE_KERNEL_LZMA=yCONFIG_KERNEL_GZIP=y
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not setCONFIG_SWAP=yCONFIG_SYSVIPC=yCONFIG_SYSVIPC_SYSCTL=yCONFIG_POSIX_MQUEUE=yCONFIG_POSIX_MQUEUE_SYSCTL=yCONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not setCONFIG_TASKSTATS=yCONFIG_TASK_DELAY_ACCT=yCONFIG_TASK_XACCT=yCONFIG_TASK_IO_ACCOUNTING=y
# CONFIG_AUDIT is not set
#
# RCU Subsystem
#
# CONFIG_CLASSIC_RCU is not set
# CONFIG_TREE_RCU is not setCONFIG_PREEMPT_RCU=yCONFIG_RCU_TRACE=y
# CONFIG_TREE_RCU_TRACE is not setCONFIG_PREEMPT_RCU_TRACE=y
# CONFIG_IKCONFIG is not setCONFIG_LOG_BUF_SHIFT=17CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=yCONFIG_GROUP_SCHED=yCONFIG_FAIR_GROUP_SCHED=yCONFIG_RT_GROUP_SCHED=y
# CONFIG_USER_SCHED is not setCONFIG_CGROUP_SCHED=yCONFIG_CGROUPS=yCONFIG_CGROUP_DEBUG=yCONFIG_CGROUP_NS=yCONFIG_CGROUP_FREEZER=yCONFIG_CGROUP_DEVICE=yCONFIG_CPUSETS=yCONFIG_PROC_PID_CPUSET=yCONFIG_CGROUP_CPUACCT=yCONFIG_RESOURCE_COUNTERS=yCONFIG_CGROUP_MEM_RES_CTLR=yCONFIG_CGROUP_MEM_RES_CTLR_SWAP=yCONFIG_GROUP_IOSCHED=yCONFIG_CGROUP_BLKIO=yCONFIG_CGROUP_PAGE=yCONFIG_MM_OWNER=yCONFIG_SYSFS_DEPRECATED=yCONFIG_SYSFS_DEPRECATED_V2=yCONFIG_RELAY=yCONFIG_NAMESPACES=y
# CONFIG_UTS_NS is not set
# CONFIG_IPC_NS is not setCONFIG_USER_NS=yCONFIG_PID_NS=y
# CONFIG_NET_NS is not setCONFIG_BLK_DEV_INITRD=yCONFIG_INITRAMFS_SOURCE=""CONFIG_RD_GZIP=yCONFIG_RD_BZIP2=yCONFIG_RD_LZMA=yCONFIG_CC_OPTIMIZE_FOR_SIZE=yCONFIG_SYSCTL=yCONFIG_ANON_INODES=y
# CONFIG_EMBEDDED is not setCONFIG_UID16=yCONFIG_SYSCTL_SYSCALL=yCONFIG_KALLSYMS=yCONFIG_KALLSYMS_ALL=yCONFIG_KALLSYMS_EXTRA_PASS=y
# CONFIG_STRIP_ASM_SYMS is not setCONFIG_HOTPLUG=yCONFIG_PRINTK=yCONFIG_BUG=yCONFIG_ELF_CORE=yCONFIG_PCSPKR_PLATFORM=yCONFIG_BASE_FULL=yCONFIG_FUTEX=yCONFIG_EPOLL=yCONFIG_SIGNALFD=yCONFIG_TIMERFD=yCONFIG_EVENTFD=yCONFIG_SHMEM=yCONFIG_AIO=yCONFIG_VM_EVENT_COUNTERS=yCONFIG_PCI_QUIRKS=yCONFIG_SLUB_DEBUG=yCONFIG_COMPAT_BRK=y
# CONFIG_SLAB is not setCONFIG_SLUB=y
# CONFIG_SLOB is not setCONFIG_PROFILING=yCONFIG_TRACEPOINTS=yCONFIG_MARKERS=yCONFIG_OPROFILE=m
# CONFIG_OPROFILE_IBS is not setCONFIG_HAVE_OPROFILE=yCONFIG_KPROBES=yCONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=yCONFIG_KRETPROBES=yCONFIG_HAVE_IOREMAP_PROT=yCONFIG_HAVE_KPROBES=yCONFIG_HAVE_KRETPROBES=yCONFIG_HAVE_ARCH_TRACEHOOK=yCONFIG_HAVE_DMA_API_DEBUG=y
# CONFIG_SLOW_WORK is not setCONFIG_HAVE_GENERIC_DMA_COHERENT=yCONFIG_SLABINFO=yCONFIG_RT_MUTEXES=yCONFIG_BASE_SMALL=0CONFIG_MODULES=y
# CONFIG_MODULE_FORCE_LOAD is not setCONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
# CONFIG_MODVERSIONS is not set
# CONFIG_MODULE_SRCVERSION_ALL is not setCONFIG_STOP_MACHINE=yCONFIG_BLOCK=yCONFIG_LBD=yCONFIG_BLK_DEV_BSG=y
# CONFIG_BLK_DEV_INTEGRITY is not set
#
# IO Schedulers
#CONFIG_ELV_FAIR_QUEUING=yCONFIG_IOSCHED_NOOP=yCONFIG_IOSCHED_NOOP_HIER=yCONFIG_IOSCHED_AS=mCONFIG_IOSCHED_AS_HIER=yCONFIG_IOSCHED_DEADLINE=mCONFIG_IOSCHED_DEADLINE_HIER=yCONFIG_IOSCHED_CFQ=yCONFIG_IOSCHED_CFQ_HIER=y
# CONFIG_DEFAULT_AS is not set
# CONFIG_DEFAULT_DEADLINE is not setCONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not setCONFIG_DEFAULT_IOSCHED="cfq"CONFIG_TRACK_ASYNC_CONTEXT=yCONFIG_DEBUG_GROUP_IOSCHED=yCONFIG_FREEZER=y
#
# Processor type and features
#CONFIG_TICK_ONESHOT=yCONFIG_NO_HZ=yCONFIG_HIGH_RES_TIMERS=y CONFIG_GENERIC_CLOCKEVENTS_BUILD=y CONFIG_SMP=y
# CONFIG_SPARSE_IRQ is not set CONFIG_X86_MPPARSE=y
# CONFIG_X86_BIGSMP is not set CONFIG_X86_EXTENDED_PLATFORM=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_RDC321X is not set
# CONFIG_X86_32_NON_STANDARD is not set CONFIG_SCHED_OMIT_FRAME_POINTER=y
# CONFIG_PARAVIRT_GUEST is not set
# CONFIG_MEMTEST is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not setCONFIG_M686=y
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MGEODEGX1 is not set
# CONFIG_MGEODE_LX is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_MVIAC7 is not set
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_GENERIC_CPU is not set CONFIG_X86_GENERIC=yCONFIG_X86_CPU=y CONFIG_X86_L1_CACHE_BYTES=64 CONFIG_X86_INTERNODE_CACHE_BYTES=64 CONFIG_X86_CMPXCHG=y CONFIG_X86_L1_CACHE_SHIFT=5 CONFIG_X86_XADD=yCONFIG_X86_PPRO_FENCE=y CONFIG_X86_WP_WORKS_OK=yCONFIG_X86_INVLPG=yCONFIG_X86_BSWAP=yCONFIG_X86_POPAD_OK=y CONFIG_X86_INTEL_USERCOPY=y CONFIG_X86_USE_PPRO_CHECKSUM=y CONFIG_X86_TSC=yCONFIG_X86_CMOV=y CONFIG_X86_MINIMUM_CPU_FAMILY=4 CONFIG_X86_DEBUGCTLMSR=y CONFIG_CPU_SUP_INTEL=y CONFIG_CPU_SUP_CYRIX_32=y CONFIG_CPU_SUP_AMD=y CONFIG_CPU_SUP_CENTAUR=y CONFIG_CPU_SUP_TRANSMETA_32=y CONFIG_CPU_SUP_UMC_32=y
# CONFIG_X86_DS is not set CONFIG_HPET_TIMER=y CONFIG_HPET_EMULATE_RTC=yCONFIG_DMI=y
# CONFIG_IOMMU_HELPER is not set
# CONFIG_IOMMU_API is not set CONFIG_NR_CPUS=8
# CONFIG_SCHED_SMT is not set CONFIG_SCHED_MC=y
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set CONFIG_PREEMPT=yCONFIG_X86_LOCAL_APIC=yCONFIG_X86_IO_APIC=y
# CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS is not set CONFIG_X86_MCE=y
# CONFIG_X86_MCE_NONFATAL is not set
# CONFIG_X86_MCE_P4THERMAL is not set CONFIG_VM86=y
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
# CONFIG_X86_REBOOTFIXUPS is not set
# CONFIG_MICROCODE is not set CONFIG_X86_MSR=mCONFIG_X86_CPUID=m
# CONFIG_X86_CPU_DEBUG is not set
# CONFIG_NOHIGHMEM is not set CONFIG_HIGHMEM4G=y
# CONFIG_HIGHMEM64G is not set CONFIG_PAGE_OFFSET=0xC0000000 CONFIG_HIGHMEM=y
# CONFIG_ARCH_PHYS_ADDR_T_64BIT is not set CONFIG_ARCH_FLATMEM_ENABLE=y CONFIG_ARCH_SPARSEMEM_ENABLE=y CONFIG_ARCH_SELECT_MEMORY_MODEL=y CONFIG_SELECT_MEMORY_MODEL=y CONFIG_FLATMEM_MANUAL=y
# CONFIG_DISCONTIGMEM_MANUAL is not set
# CONFIG_SPARSEMEM_MANUAL is not set CONFIG_FLATMEM=yCONFIG_FLAT_NODE_MEM_MAP=y CONFIG_SPARSEMEM_STATIC=y CONFIG_PAGEFLAGS_EXTENDED=y CONFIG_SPLIT_PTLOCK_CPUS=4
# CONFIG_PHYS_ADDR_T_64BIT is not set CONFIG_ZONE_DMA_FLAG=1CONFIG_BOUNCE=yCONFIG_VIRT_TO_BUS=y CONFIG_UNEVICTABLE_LRU=yCONFIG_HAVE_MLOCK=y CONFIG_HAVE_MLOCKED_PAGE_BIT=y CONFIG_HIGHPTE=y
# CONFIG_X86_CHECK_BIOS_CORRUPTION is not set CONFIG_X86_RESERVE_LOW_64K=y
# CONFIG_MATH_EMULATION is not set CONFIG_MTRR=yCONFIG_MTRR_SANITIZER=y CONFIG_MTRR_SANITIZER_ENABLE_DEFAULT=0 CONFIG_MTRR_SANITIZER_SPARE_REG_NR_DEFAULT=1 # CONFIG_X86_PAT is not setCONFIG_EFI=yCONFIG_SECCOMP=y
# CONFIG_CC_STACKPROTECTOR is not set
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not setCONFIG_HZ_1000=yCONFIG_HZ=1000CONFIG_SCHED_HRTICK=yCONFIG_KEXEC=yCONFIG_CRASH_DUMP=y CONFIG_PHYSICAL_START=0x1000000 CONFIG_RELOCATABLE=y CONFIG_PHYSICAL_ALIGN=0x400000 CONFIG_HOTPLUG_CPU=y
# CONFIG_COMPAT_VDSO is not set
# CONFIG_CMDLINE_BOOL is not set CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
#
# Power management and ACPI options
#CONFIG_PM=yCONFIG_PM_DEBUG=y
# CONFIG_PM_VERBOSE is not setCONFIG_CAN_PM_TRACE=y
# CONFIG_PM_TRACE_RTC is not setCONFIG_PM_SLEEP_SMP=yCONFIG_PM_SLEEP=yCONFIG_SUSPEND=yCONFIG_SUSPEND_FREEZER=y
# CONFIG_HIBERNATION is not setCONFIG_ACPI=yCONFIG_ACPI_SLEEP=y
# CONFIG_ACPI_PROCFS is not set
# CONFIG_ACPI_PROCFS_POWER is not setCONFIG_ACPI_SYSFS_POWER=y
# CONFIG_ACPI_PROC_EVENT is not setCONFIG_ACPI_AC=m
# CONFIG_ACPI_BATTERY is not setCONFIG_ACPI_BUTTON=mCONFIG_ACPI_VIDEO=mCONFIG_ACPI_FAN=yCONFIG_ACPI_DOCK=yCONFIG_ACPI_PROCESSOR=yCONFIG_ACPI_HOTPLUG_CPU=yCONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_CUSTOM_DSDT is not setCONFIG_ACPI_BLACKLIST_YEAR=1999
# CONFIG_ACPI_DEBUG is not set
# CONFIG_ACPI_PCI_SLOT is not setCONFIG_X86_PM_TIMER=yCONFIG_ACPI_CONTAINER=y
# CONFIG_ACPI_SBS is not setCONFIG_X86_APM_BOOT=yCONFIG_APM=y
# CONFIG_APM_IGNORE_USER_SUSPEND is not set
# CONFIG_APM_DO_ENABLE is not setCONFIG_APM_CPU_IDLE=y
# CONFIG_APM_DISPLAY_BLANK is not set
# CONFIG_APM_ALLOW_INTS is not set
#
# CPU Frequency scaling
#CONFIG_CPU_FREQ=yCONFIG_CPU_FREQ_TABLE=y CONFIG_CPU_FREQ_DEBUG=y CONFIG_CPU_FREQ_STAT=m CONFIG_CPU_FREQ_STAT_DETAILS=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set # CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set # CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set CONFIG_CPU_FREQ_GOV_PERFORMANCE=y CONFIG_CPU_FREQ_GOV_POWERSAVE=m CONFIG_CPU_FREQ_GOV_USERSPACE=y CONFIG_CPU_FREQ_GOV_ONDEMAND=m CONFIG_CPU_FREQ_GOV_CONSERVATIVE=m
#
# CPUFreq processor drivers
#
# CONFIG_X86_ACPI_CPUFREQ is not set
# CONFIG_X86_POWERNOW_K6 is not set
# CONFIG_X86_POWERNOW_K7 is not set
# CONFIG_X86_POWERNOW_K8 is not set
# CONFIG_X86_GX_SUSPMOD is not set
# CONFIG_X86_SPEEDSTEP_CENTRINO is not setCONFIG_X86_SPEEDSTEP_ICH=yCONFIG_X86_SPEEDSTEP_SMI=y
# CONFIG_X86_P4_CLOCKMOD is not set
# CONFIG_X86_CPUFREQ_NFORCE2 is not set
# CONFIG_X86_LONGRUN is not set
# CONFIG_X86_LONGHAUL is not set
# CONFIG_X86_E_POWERSAVER is not set
#
# shared options # CONFIG_X86_SPEEDSTEP_LIB=y # CONFIG_X86_SPEEDSTEP_RELAXED_CAP_CHECK is not set CONFIG_CPU_IDLE=y CONFIG_CPU_IDLE_GOV_LADDER=y CONFIG_CPU_IDLE_GOV_MENU=y
#
# Bus options (PCI etc.)
#CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
# CONFIG_PCI_GOOLPC is not setCONFIG_PCI_GOANY=yCONFIG_PCI_BIOS=yCONFIG_PCI_DIRECT=yCONFIG_PCI_MMCONFIG=yCONFIG_PCI_DOMAINS=yCONFIG_PCIEPORTBUS=yCONFIG_HOTPLUG_PCI_PCIE=mCONFIG_PCIEAER=y
# CONFIG_PCIEASPM is not setCONFIG_ARCH_SUPPORTS_MSI=y
# CONFIG_PCI_MSI is not setCONFIG_PCI_LEGACY=y
# CONFIG_PCI_DEBUG is not set
# CONFIG_PCI_STUB is not setCONFIG_HT_IRQ=y
# CONFIG_PCI_IOV is not setCONFIG_ISA_DMA_API=yCONFIG_ISA=y
# CONFIG_EISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set
# CONFIG_OLPC is not setCONFIG_PCCARD=y
# CONFIG_PCMCIA_DEBUG is not setCONFIG_PCMCIA=yCONFIG_PCMCIA_LOAD_CIS=y
# CONFIG_PCMCIA_IOCTL is not setCONFIG_CARDBUS=y
#
# PC-card bridges
#CONFIG_YENTA=yCONFIG_YENTA_O2=yCONFIG_YENTA_RICOH=yCONFIG_YENTA_TI=yCONFIG_YENTA_ENE_TUNE=yCONFIG_YENTA_TOSHIBA=y
# CONFIG_PD6729 is not set
# CONFIG_I82092 is not set
# CONFIG_I82365 is not set
# CONFIG_TCIC is not setCONFIG_PCMCIA_PROBE=yCONFIG_PCCARD_NONSTATIC=yCONFIG_HOTPLUG_PCI=yCONFIG_HOTPLUG_PCI_FAKE=m
# CONFIG_HOTPLUG_PCI_COMPAQ is not set
# CONFIG_HOTPLUG_PCI_IBM is not setCONFIG_HOTPLUG_PCI_ACPI=mCONFIG_HOTPLUG_PCI_ACPI_IBM=m
# CONFIG_HOTPLUG_PCI_CPCI is not set
# CONFIG_HOTPLUG_PCI_SHPC is not set
#
# Executable file formats / Emulations
#CONFIG_BINFMT_ELF=y
# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not setCONFIG_HAVE_AOUT=y
# CONFIG_BINFMT_AOUT is not setCONFIG_BINFMT_MISC=yCONFIG_HAVE_ATOMIC_IOMAP=yCONFIG_NET=y
#
# Networking options
#CONFIG_PACKET=yCONFIG_PACKET_MMAP=yCONFIG_UNIX=y
# CONFIG_NET_KEY is not setCONFIG_INET=yCONFIG_IP_MULTICAST=yCONFIG_IP_ADVANCED_ROUTER=yCONFIG_ASK_IP_FIB_HASH=y
# CONFIG_IP_FIB_TRIE is not setCONFIG_IP_FIB_HASH=yCONFIG_IP_MULTIPLE_TABLES=yCONFIG_IP_ROUTE_MULTIPATH=yCONFIG_IP_ROUTE_VERBOSE=y
# CONFIG_IP_PNP is not setCONFIG_NET_IPIP=m
# CONFIG_NET_IPGRE is not setCONFIG_IP_MROUTE=yCONFIG_IP_PIMSM_V1=yCONFIG_IP_PIMSM_V2=y
# CONFIG_ARPD is not setCONFIG_SYN_COOKIES=y
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
# CONFIG_INET_XFRM_TUNNEL is not setCONFIG_INET_TUNNEL=m
# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not setCONFIG_INET_LRO=mCONFIG_INET_DIAG=mCONFIG_INET_TCP_DIAG=mCONFIG_TCP_CONG_ADVANCED=yCONFIG_TCP_CONG_BIC=mCONFIG_TCP_CONG_CUBIC=y
# CONFIG_TCP_CONG_WESTWOOD is not set
# CONFIG_TCP_CONG_HTCP is not setCONFIG_TCP_CONG_HSTCP=mCONFIG_TCP_CONG_HYBLA=m
# CONFIG_TCP_CONG_VEGAS is not setCONFIG_TCP_CONG_SCALABLE=mCONFIG_TCP_CONG_LP=m
# CONFIG_TCP_CONG_VENO is not set
# CONFIG_TCP_CONG_YEAH is not setCONFIG_TCP_CONG_ILLINOIS=m
# CONFIG_DEFAULT_BIC is not setCONFIG_DEFAULT_CUBIC=y
# CONFIG_DEFAULT_HTCP is not set
# CONFIG_DEFAULT_VEGAS is not set
# CONFIG_DEFAULT_WESTWOOD is not set
# CONFIG_DEFAULT_RENO is not setCONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set
# CONFIG_IPV6 is not set
# CONFIG_NETWORK_SECMARK is not set
# CONFIG_NETFILTER is not set
# CONFIG_IP_DCCP is not set
# CONFIG_IP_SCTP is not set
# CONFIG_TIPC is not set
# CONFIG_ATM is not setCONFIG_STP=mCONFIG_BRIDGE=m
# CONFIG_NET_DSA is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not setCONFIG_LLC=m
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_PHONET is not setCONFIG_NET_SCHED=y
#
# Queueing/Scheduling
#
# CONFIG_NET_SCH_CBQ is not set
# CONFIG_NET_SCH_HTB is not set
# CONFIG_NET_SCH_HFSC is not set
# CONFIG_NET_SCH_PRIO is not set
# CONFIG_NET_SCH_MULTIQ is not set
# CONFIG_NET_SCH_RED is not set
# CONFIG_NET_SCH_SFQ is not set
# CONFIG_NET_SCH_TEQL is not set
# CONFIG_NET_SCH_TBF is not set
# CONFIG_NET_SCH_GRED is not set
# CONFIG_NET_SCH_DSMARK is not set
# CONFIG_NET_SCH_NETEM is not set
# CONFIG_NET_SCH_DRR is not set
#
# Classification
#CONFIG_NET_CLS=y
# CONFIG_NET_CLS_BASIC is not set
# CONFIG_NET_CLS_TCINDEX is not set
# CONFIG_NET_CLS_ROUTE4 is not set
# CONFIG_NET_CLS_FW is not set
# CONFIG_NET_CLS_U32 is not set
# CONFIG_NET_CLS_RSVP is not set
# CONFIG_NET_CLS_RSVP6 is not set
# CONFIG_NET_CLS_FLOW is not setCONFIG_NET_CLS_CGROUP=y
# CONFIG_NET_EMATCH is not set
# CONFIG_NET_CLS_ACT is not setCONFIG_NET_SCH_FIFO=y
# CONFIG_DCB is not set
#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_NET_TCPPROBE is not set
# CONFIG_NET_DROP_MONITOR is not set
# CONFIG_HAMRADIO is not set
# CONFIG_CAN is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_AF_RXRPC is not setCONFIG_FIB_RULES=y
# CONFIG_WIRELESS is not set
# CONFIG_WIMAX is not set
# CONFIG_RFKILL is not set
# CONFIG_NET_9P is not set
#
# Device Drivers
#
#
# Generic Driver Options
#CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"CONFIG_STANDALONE=yCONFIG_PREVENT_FIRMWARE_BUILD=yCONFIG_FW_LOADER=yCONFIG_FIRMWARE_IN_KERNEL=yCONFIG_EXTRA_FIRMWARE=""
# CONFIG_DEBUG_DRIVER is not setCONFIG_DEBUG_DEVRES=y
# CONFIG_SYS_HYPERVISOR is not set
# CONFIG_CONNECTOR is not set
# CONFIG_MTD is not setCONFIG_PARPORT=mCONFIG_PARPORT_PC=mCONFIG_PARPORT_SERIAL=m
# CONFIG_PARPORT_PC_FIFO is not set
# CONFIG_PARPORT_PC_SUPERIO is not setCONFIG_PARPORT_PC_PCMCIA=m
# CONFIG_PARPORT_GSC is not set
# CONFIG_PARPORT_AX88796 is not setCONFIG_PARPORT_1284=yCONFIG_PNP=yCONFIG_PNP_DEBUG_MESSAGES=y
#
# Protocols
#CONFIG_ISAPNP=y
# CONFIG_PNPBIOS is not setCONFIG_PNPACPI=yCONFIG_BLK_DEV=y
# CONFIG_BLK_DEV_FD is not set
# CONFIG_BLK_DEV_XD is not setCONFIG_PARIDE=m
#
# Parallel IDE high-level drivers
#CONFIG_PARIDE_PD=mCONFIG_PARIDE_PCD=mCONFIG_PARIDE_PF=m
# CONFIG_PARIDE_PT is not setCONFIG_PARIDE_PG=m
#
# Parallel IDE protocol modules
#
# CONFIG_PARIDE_ATEN is not set
# CONFIG_PARIDE_BPCK is not set
# CONFIG_PARIDE_BPCK6 is not set
# CONFIG_PARIDE_COMM is not set
# CONFIG_PARIDE_DSTR is not set
# CONFIG_PARIDE_FIT2 is not set
# CONFIG_PARIDE_FIT3 is not set
# CONFIG_PARIDE_EPAT is not set
# CONFIG_PARIDE_EPIA is not set
# CONFIG_PARIDE_FRIQ is not set
# CONFIG_PARIDE_FRPW is not set
# CONFIG_PARIDE_KBIC is not set
# CONFIG_PARIDE_KTTI is not set
# CONFIG_PARIDE_ON20 is not set
# CONFIG_PARIDE_ON26 is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not setCONFIG_BLK_DEV_LOOP=mCONFIG_BLK_DEV_CRYPTOLOOP=mCONFIG_BLK_DEV_NBD=m
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not setCONFIG_BLK_DEV_RAM=yCONFIG_BLK_DEV_RAM_COUNT=16CONFIG_BLK_DEV_RAM_SIZE=16384
# CONFIG_BLK_DEV_XIP is not set
# CONFIG_CDROM_PKTCDVD is not set
# CONFIG_ATA_OVER_ETH is not set
# CONFIG_BLK_DEV_HD is not setCONFIG_MISC_DEVICES=y
# CONFIG_IBM_ASM is not set
# CONFIG_PHANTOM is not set
# CONFIG_SGI_IOC4 is not set
# CONFIG_TIFM_CORE is not set
# CONFIG_ICS932S401 is not set
# CONFIG_ENCLOSURE_SERVICES is not set
# CONFIG_HP_ILO is not set
# CONFIG_ISL29003 is not set
# CONFIG_C2PORT is not set
#
# EEPROM support
#
# CONFIG_EEPROM_AT24 is not set
# CONFIG_EEPROM_LEGACY is not setCONFIG_EEPROM_93CX6=mCONFIG_HAVE_IDE=y
# CONFIG_IDE is not set
#
# SCSI device support
#
# CONFIG_RAID_ATTRS is not setCONFIG_SCSI=mCONFIG_SCSI_DMA=yCONFIG_SCSI_TGT=mCONFIG_SCSI_NETLINK=yCONFIG_SCSI_PROC_FS=y
#
# SCSI support type (disk, tape, CD-ROM)
#CONFIG_BLK_DEV_SD=m
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not setCONFIG_BLK_DEV_SR=mCONFIG_BLK_DEV_SR_VENDOR=yCONFIG_CHR_DEV_SG=mCONFIG_CHR_DEV_SCH=m
#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs #CONFIG_SCSI_MULTI_LUN=y
# CONFIG_SCSI_CONSTANTS is not set CONFIG_SCSI_LOGGING=yCONFIG_SCSI_SCAN_ASYNC=yCONFIG_SCSI_WAIT_SCAN=m
#
# SCSI Transports
#CONFIG_SCSI_SPI_ATTRS=mCONFIG_SCSI_FC_ATTRS=m
# CONFIG_SCSI_FC_TGT_ATTRS is not setCONFIG_SCSI_ISCSI_ATTRS=mCONFIG_SCSI_SAS_ATTRS=mCONFIG_SCSI_SAS_LIBSAS=mCONFIG_SCSI_SAS_ATA=yCONFIG_SCSI_SAS_HOST_SMP=y
# CONFIG_SCSI_SAS_LIBSAS_DEBUG is not setCONFIG_SCSI_SRP_ATTRS=m
# CONFIG_SCSI_SRP_TGT_ATTRS is not setCONFIG_SCSI_LOWLEVEL=yCONFIG_ISCSI_TCP=m
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_7000FASST is not setCONFIG_SCSI_ACARD=m
# CONFIG_SCSI_AHA152X is not set
# CONFIG_SCSI_AHA1542 is not set
# CONFIG_SCSI_AACRAID is not setCONFIG_SCSI_AIC7XXX=mCONFIG_AIC7XXX_CMDS_PER_DEVICE=4CONFIG_AIC7XXX_RESET_DELAY_MS=15000
# CONFIG_AIC7XXX_DEBUG_ENABLE is not setCONFIG_AIC7XXX_DEBUG_MASK=0
# CONFIG_AIC7XXX_REG_PRETTY_PRINT is not setCONFIG_SCSI_AIC7XXX_OLD=mCONFIG_SCSI_AIC79XX=mCONFIG_AIC79XX_CMDS_PER_DEVICE=4CONFIG_AIC79XX_RESET_DELAY_MS=15000
# CONFIG_AIC79XX_DEBUG_ENABLE is not setCONFIG_AIC79XX_DEBUG_MASK=0
# CONFIG_AIC79XX_REG_PRETTY_PRINT is not setCONFIG_SCSI_AIC94XX=m
# CONFIG_AIC94XX_DEBUG is not set
# CONFIG_SCSI_DPT_I2O is not setCONFIG_SCSI_ADVANSYS=m
# CONFIG_SCSI_IN2000 is not set
# CONFIG_SCSI_ARCMSR is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
# CONFIG_MEGARAID_SAS is not set
# CONFIG_SCSI_MPT2SAS is not set
# CONFIG_SCSI_HPTIOP is not setCONFIG_SCSI_BUSLOGIC=m
# CONFIG_SCSI_FLASHPOINT is not set
# CONFIG_LIBFC is not set
# CONFIG_LIBFCOE is not set
# CONFIG_FCOE is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_DTC3280 is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not setCONFIG_SCSI_GDTH=m
# CONFIG_SCSI_GENERIC_NCR5380 is not set
# CONFIG_SCSI_GENERIC_NCR5380_MMIO is not setCONFIG_SCSI_IPS=mCONFIG_SCSI_INITIO=mCONFIG_SCSI_INIA100=mCONFIG_SCSI_PPA=mCONFIG_SCSI_IMM=m
# CONFIG_SCSI_IZIP_EPP16 is not set
# CONFIG_SCSI_IZIP_SLOW_CTR is not set
# CONFIG_SCSI_MVSAS is not set
# CONFIG_SCSI_NCR53C406A is not set
# CONFIG_SCSI_STEX is not setCONFIG_SCSI_SYM53C8XX_2=mCONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64CONFIG_SCSI_SYM53C8XX_MMIO=y
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_PAS16 is not set
# CONFIG_SCSI_QLOGIC_FAS is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
# CONFIG_SCSI_QLA_FC is not set
# CONFIG_SCSI_QLA_ISCSI is not set
# CONFIG_SCSI_LPFC is not set
# CONFIG_SCSI_SYM53C416 is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_T128 is not set
# CONFIG_SCSI_U14_34F is not set
# CONFIG_SCSI_ULTRASTOR is not set
# CONFIG_SCSI_NSP32 is not set
# CONFIG_SCSI_DEBUG is not set
# CONFIG_SCSI_SRP is not setCONFIG_SCSI_LOWLEVEL_PCMCIA=y
# CONFIG_PCMCIA_AHA152X is not set
# CONFIG_PCMCIA_FDOMAIN is not set
# CONFIG_PCMCIA_NINJA_SCSI is not setCONFIG_PCMCIA_QLOGIC=m
# CONFIG_PCMCIA_SYM53C500 is not set
# CONFIG_SCSI_DH is not set
# CONFIG_SCSI_OSD_INITIATOR is not setCONFIG_ATA=m
# CONFIG_ATA_NONSTANDARD is not setCONFIG_ATA_ACPI=yCONFIG_SATA_PMP=yCONFIG_SATA_AHCI=m
# CONFIG_SATA_SIL24 is not setCONFIG_ATA_SFF=y
# CONFIG_SATA_SVW is not setCONFIG_ATA_PIIX=m
# CONFIG_SATA_MV is not setCONFIG_SATA_NV=m
# CONFIG_PDC_ADMA is not set
# CONFIG_SATA_QSTOR is not set
# CONFIG_SATA_PROMISE is not set
# CONFIG_SATA_SX4 is not set
# CONFIG_SATA_SIL is not setCONFIG_SATA_SIS=m
# CONFIG_SATA_ULI is not set
# CONFIG_SATA_VIA is not set
# CONFIG_SATA_VITESSE is not set
# CONFIG_SATA_INIC162X is not set
# CONFIG_PATA_ACPI is not set
# CONFIG_PATA_ALI is not set
# CONFIG_PATA_AMD is not set
# CONFIG_PATA_ARTOP is not setCONFIG_PATA_ATIIXP=m
# CONFIG_PATA_CMD640_PCI is not set
# CONFIG_PATA_CMD64X is not set
# CONFIG_PATA_CS5520 is not set
# CONFIG_PATA_CS5530 is not set
# CONFIG_PATA_CS5535 is not set
# CONFIG_PATA_CS5536 is not set
# CONFIG_PATA_CYPRESS is not set
# CONFIG_PATA_EFAR is not setCONFIG_ATA_GENERIC=m
# CONFIG_PATA_HPT366 is not set
# CONFIG_PATA_HPT37X is not set
# CONFIG_PATA_HPT3X2N is not set
# CONFIG_PATA_HPT3X3 is not set
# CONFIG_PATA_ISAPNP is not set
# CONFIG_PATA_IT821X is not set
# CONFIG_PATA_IT8213 is not set
# CONFIG_PATA_JMICRON is not set
# CONFIG_PATA_LEGACY is not set
# CONFIG_PATA_TRIFLEX is not set
# CONFIG_PATA_MARVELL is not setCONFIG_PATA_MPIIX=m
# CONFIG_PATA_OLDPIIX is not set
# CONFIG_PATA_NETCELL is not set
# CONFIG_PATA_NINJA32 is not set
# CONFIG_PATA_NS87410 is not set
# CONFIG_PATA_NS87415 is not set
# CONFIG_PATA_OPTI is not set
# CONFIG_PATA_OPTIDMA is not setCONFIG_PATA_PCMCIA=m
# CONFIG_PATA_PDC_OLD is not set
# CONFIG_PATA_QDI is not set
# CONFIG_PATA_RADISYS is not set
# CONFIG_PATA_RZ1000 is not set
# CONFIG_PATA_SC1200 is not set
# CONFIG_PATA_SERVERWORKS is not set
# CONFIG_PATA_PDC2027X is not set
# CONFIG_PATA_SIL680 is not setCONFIG_PATA_SIS=mCONFIG_PATA_VIA=m
# CONFIG_PATA_WINBOND is not set
# CONFIG_PATA_WINBOND_VLB is not set
# CONFIG_PATA_SCH is not set
# CONFIG_MD is not setCONFIG_FUSION=yCONFIG_FUSION_SPI=mCONFIG_FUSION_FC=m
# CONFIG_FUSION_SAS is not setCONFIG_FUSION_MAX_SGE=40CONFIG_FUSION_CTL=mCONFIG_FUSION_LAN=mCONFIG_FUSION_LOGGING=y
#
# IEEE 1394 (FireWire) support
#
#
# Enable only one of the two stacks, unless you know what you are doing #CONFIG_FIREWIRE=mCONFIG_FIREWIRE_OHCI=mCONFIG_FIREWIRE_OHCI_DEBUG=yCONFIG_FIREWIRE_SBP2=m
# CONFIG_IEEE1394 is not setCONFIG_I2O=m
# CONFIG_I2O_LCT_NOTIFY_ON_CHANGES is not set CONFIG_I2O_EXT_ADAPTEC=yCONFIG_I2O_CONFIG=mCONFIG_I2O_CONFIG_OLD_IOCTL=yCONFIG_I2O_BUS=mCONFIG_I2O_BLOCK=mCONFIG_I2O_SCSI=mCONFIG_I2O_PROC=m
# CONFIG_MACINTOSH_DRIVERS is not setCONFIG_NETDEVICES=yCONFIG_COMPAT_NET_DEV_OPS=yCONFIG_DUMMY=mCONFIG_BONDING=m
# CONFIG_MACVLAN is not set
# CONFIG_EQUALIZER is not setCONFIG_TUN=m
# CONFIG_VETH is not set
# CONFIG_NET_SB1000 is not set
# CONFIG_ARCNET is not setCONFIG_PHYLIB=m
#
# MII PHY device drivers
#
# CONFIG_MARVELL_PHY is not set
# CONFIG_DAVICOM_PHY is not set
# CONFIG_QSEMI_PHY is not set CONFIG_LXT_PHY=m
# CONFIG_CICADA_PHY is not set
# CONFIG_VITESSE_PHY is not set
# CONFIG_SMSC_PHY is not set
# CONFIG_BROADCOM_PHY is not set
# CONFIG_ICPLUS_PHY is not set
# CONFIG_REALTEK_PHY is not set
# CONFIG_NATIONAL_PHY is not set
# CONFIG_STE10XP is not set
# CONFIG_LSI_ET1011C_PHY is not set
# CONFIG_MDIO_BITBANG is not set CONFIG_NET_ETHERNET=yCONFIG_MII=m
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
# CONFIG_CASSINI is not set CONFIG_NET_VENDOR_3COM=y
# CONFIG_EL1 is not set
# CONFIG_EL2 is not set
# CONFIG_ELPLUS is not set
# CONFIG_EL16 is not setCONFIG_EL3=m
# CONFIG_3C515 is not setCONFIG_VORTEX=mCONFIG_TYPHOON=m
# CONFIG_LANCE is not set CONFIG_NET_VENDOR_SMC=y
# CONFIG_WD80x3 is not set
# CONFIG_ULTRA is not set
# CONFIG_SMC9194 is not set
# CONFIG_ETHOC is not set
# CONFIG_NET_VENDOR_RACAL is not set
# CONFIG_DNET is not set CONFIG_NET_TULIP=yCONFIG_DE2104X=mCONFIG_TULIP=m
# CONFIG_TULIP_MWI is not set CONFIG_TULIP_MMIO=y
# CONFIG_TULIP_NAPI is not set CONFIG_DE4X5=mCONFIG_WINBOND_840=mCONFIG_DM9102=mCONFIG_ULI526X=mCONFIG_PCMCIA_XIRCOM=m
# CONFIG_AT1700 is not set
# CONFIG_DEPCA is not set
# CONFIG_HP100 is not setCONFIG_NET_ISA=y
# CONFIG_E2100 is not set
# CONFIG_EWRK3 is not set
# CONFIG_EEXPRESS is not set
# CONFIG_EEXPRESS_PRO is not set
# CONFIG_HPLAN_PLUS is not set
# CONFIG_HPLAN is not set
# CONFIG_LP486E is not set
# CONFIG_ETH16I is not setCONFIG_NE2000=m
# CONFIG_ZNET is not set
# CONFIG_SEEQ8005 is not set
# CONFIG_IBM_NEW_EMAC_ZMII is not set
# CONFIG_IBM_NEW_EMAC_RGMII is not set
# CONFIG_IBM_NEW_EMAC_TAH is not set
# CONFIG_IBM_NEW_EMAC_EMAC4 is not set
# CONFIG_IBM_NEW_EMAC_NO_FLOW_CTRL is not set # CONFIG_IBM_NEW_EMAC_MAL_CLR_ICINTSTAT is not set # CONFIG_IBM_NEW_EMAC_MAL_COMMON_ERR is not set CONFIG_NET_PCI=yCONFIG_PCNET32=mCONFIG_AMD8111_ETH=m CONFIG_ADAPTEC_STARFIRE=m
# CONFIG_AC3200 is not set
# CONFIG_APRICOT is not setCONFIG_B44=mCONFIG_B44_PCI_AUTOSELECT=y CONFIG_B44_PCICORE_AUTOSELECT=y CONFIG_B44_PCI=yCONFIG_FORCEDETH=mCONFIG_FORCEDETH_NAPI=y
# CONFIG_CS89x0 is not setCONFIG_E100=m
# CONFIG_FEALNX is not set
# CONFIG_NATSEMI is not set CONFIG_NE2K_PCI=m
# CONFIG_8139CP is not set CONFIG_8139TOO=m
# CONFIG_8139TOO_PIO is not set
# CONFIG_8139TOO_TUNE_TWISTER is not set CONFIG_8139TOO_8129=y
# CONFIG_8139_OLD_RX_RESET is not set
# CONFIG_R6040 is not setCONFIG_SIS900=m
# CONFIG_EPIC100 is not set
# CONFIG_SMSC9420 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_TLAN is not set CONFIG_VIA_RHINE=mCONFIG_VIA_RHINE_MMIO=y
# CONFIG_SC92031 is not set CONFIG_NET_POCKET=yCONFIG_ATP=mCONFIG_DE600=mCONFIG_DE620=m
# CONFIG_ATL2 is not set CONFIG_NETDEV_1000=yCONFIG_ACENIC=m
# CONFIG_ACENIC_OMIT_TIGON_I is not set
# CONFIG_DL2K is not setCONFIG_E1000=mCONFIG_E1000E=m
# CONFIG_IP1000 is not set
# CONFIG_IGB is not set
# CONFIG_IGBVF is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set CONFIG_R8169=m
# CONFIG_SIS190 is not setCONFIG_SKGE=m
# CONFIG_SKGE_DEBUG is not set CONFIG_SKY2=m
# CONFIG_SKY2_DEBUG is not set CONFIG_VIA_VELOCITY=m
# CONFIG_TIGON3 is not set
# CONFIG_BNX2 is not set
# CONFIG_QLA3XXX is not set
# CONFIG_ATL1 is not set
# CONFIG_ATL1E is not set
# CONFIG_ATL1C is not set
# CONFIG_JME is not set
# CONFIG_NETDEV_10000 is not set
# CONFIG_TR is not set
#
# Wireless LAN
#
# CONFIG_WLAN_PRE80211 is not set
# CONFIG_WLAN_80211 is not set
#
# Enable WiMAX (Networking options) to see the WiMAX drivers #
#
# USB Network Adapters
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not setCONFIG_USB_USBNET=mCONFIG_USB_NET_AX8817X=mCONFIG_USB_NET_CDCETHER=mCONFIG_USB_NET_DM9601=m
# CONFIG_USB_NET_SMSC95XX is not setCONFIG_USB_NET_GL620A=mCONFIG_USB_NET_NET1080=m
# CONFIG_USB_NET_PLUSB is not set
# CONFIG_USB_NET_MCS7830 is not set
# CONFIG_USB_NET_RNDIS_HOST is not setCONFIG_USB_NET_CDC_SUBSET=mCONFIG_USB_ALI_M5632=yCONFIG_USB_AN2720=yCONFIG_USB_BELKIN=yCONFIG_USB_ARMLINUX=yCONFIG_USB_EPSON2888=yCONFIG_USB_KC2190=y
# CONFIG_USB_NET_ZAURUS is not setCONFIG_NET_PCMCIA=y
# CONFIG_PCMCIA_3C589 is not set
# CONFIG_PCMCIA_3C574 is not set
# CONFIG_PCMCIA_FMVJ18X is not setCONFIG_PCMCIA_PCNET=mCONFIG_PCMCIA_NMCLAN=mCONFIG_PCMCIA_SMC91C92=m
# CONFIG_PCMCIA_XIRC2PS is not set
# CONFIG_PCMCIA_AXNET is not set
# CONFIG_WAN is not setCONFIG_FDDI=y
# CONFIG_DEFXX is not set
# CONFIG_SKFP is not set
# CONFIG_HIPPI is not setCONFIG_PLIP=mCONFIG_PPP=mCONFIG_PPP_MULTILINK=yCONFIG_PPP_FILTER=yCONFIG_PPP_ASYNC=mCONFIG_PPP_SYNC_TTY=mCONFIG_PPP_DEFLATE=m
# CONFIG_PPP_BSDCOMP is not set
# CONFIG_PPP_MPPE is not setCONFIG_PPPOE=m
# CONFIG_PPPOL2TP is not setCONFIG_SLIP=mCONFIG_SLIP_COMPRESSED=yCONFIG_SLHC=mCONFIG_SLIP_SMART=y
# CONFIG_SLIP_MODE_SLIP6 is not setCONFIG_NET_FC=yCONFIG_NETCONSOLE=m
# CONFIG_NETCONSOLE_DYNAMIC is not setCONFIG_NETPOLL=yCONFIG_NETPOLL_TRAP=yCONFIG_NET_POLL_CONTROLLER=y
# CONFIG_ISDN is not set
# CONFIG_PHONE is not set
#
# Input device support
#CONFIG_INPUT=yCONFIG_INPUT_FF_MEMLESS=yCONFIG_INPUT_POLLDEV=m
#
# Userland interfaces
#CONFIG_INPUT_MOUSEDEV=y
# CONFIG_INPUT_MOUSEDEV_PSAUX is not setCONFIG_INPUT_MOUSEDEV_SCREEN_X=1024CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not setCONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_EVBUG is not set
#
# Input Device Drivers
#CONFIG_INPUT_KEYBOARD=yCONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_STOWAWAY is not setCONFIG_INPUT_MOUSE=yCONFIG_MOUSE_PS2=yCONFIG_MOUSE_PS2_ALPS=yCONFIG_MOUSE_PS2_LOGIPS2PP=yCONFIG_MOUSE_PS2_SYNAPTICS=yCONFIG_MOUSE_PS2_LIFEBOOK=yCONFIG_MOUSE_PS2_TRACKPOINT=y
# CONFIG_MOUSE_PS2_ELANTECH is not set
# CONFIG_MOUSE_PS2_TOUCHKIT is not setCONFIG_MOUSE_SERIAL=mCONFIG_MOUSE_APPLETOUCH=m
# CONFIG_MOUSE_BCM5974 is not set
# CONFIG_MOUSE_INPORT is not set
# CONFIG_MOUSE_LOGIBM is not set
# CONFIG_MOUSE_PC110PAD is not setCONFIG_MOUSE_VSXXXAA=m
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not setCONFIG_INPUT_MISC=y
# CONFIG_INPUT_PCSPKR is not set
# CONFIG_INPUT_APANEL is not set
# CONFIG_INPUT_WISTRON_BTNS is not set
# CONFIG_INPUT_ATLAS_BTNS is not set
# CONFIG_INPUT_ATI_REMOTE is not set
# CONFIG_INPUT_ATI_REMOTE2 is not set
# CONFIG_INPUT_KEYSPAN_REMOTE is not set
# CONFIG_INPUT_POWERMATE is not set
# CONFIG_INPUT_YEALINK is not set
# CONFIG_INPUT_CM109 is not setCONFIG_INPUT_UINPUT=m
#
# Hardware I/O ports
#CONFIG_SERIO=yCONFIG_SERIO_I8042=yCONFIG_SERIO_SERPORT=y
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PARKBD is not set
# CONFIG_SERIO_PCIPS2 is not setCONFIG_SERIO_LIBPS2=yCONFIG_SERIO_RAW=m
# CONFIG_GAMEPORT is not set
#
# Character devices
#CONFIG_VT=yCONFIG_CONSOLE_TRANSLATIONS=yCONFIG_VT_CONSOLE=yCONFIG_HW_CONSOLE=yCONFIG_VT_HW_CONSOLE_BINDING=yCONFIG_DEVKMEM=yCONFIG_SERIAL_NONSTANDARD=y
# CONFIG_COMPUTONE is not setCONFIG_ROCKETPORT=mCONFIG_CYCLADES=m
# CONFIG_CYZ_INTR is not set
# CONFIG_DIGIEPCA is not set
# CONFIG_MOXA_INTELLIO is not set
# CONFIG_MOXA_SMARTIO is not set
# CONFIG_ISI is not set
# CONFIG_SYNCLINK is not setCONFIG_SYNCLINKMP=mCONFIG_SYNCLINK_GT=m
# CONFIG_N_HDLC is not set
# CONFIG_RISCOM8 is not set
# CONFIG_SPECIALIX is not set
# CONFIG_SX is not set
# CONFIG_RIO is not set
# CONFIG_STALDRV is not set
# CONFIG_NOZOMI is not set
#
# Serial drivers
#CONFIG_SERIAL_8250=yCONFIG_SERIAL_8250_CONSOLE=yCONFIG_FIX_EARLYCON_MEM=yCONFIG_SERIAL_8250_PCI=yCONFIG_SERIAL_8250_PNP=yCONFIG_SERIAL_8250_CS=mCONFIG_SERIAL_8250_NR_UARTS=32CONFIG_SERIAL_8250_RUNTIME_UARTS=4CONFIG_SERIAL_8250_EXTENDED=yCONFIG_SERIAL_8250_MANY_PORTS=y
# CONFIG_SERIAL_8250_FOURPORT is not set
# CONFIG_SERIAL_8250_ACCENT is not set
# CONFIG_SERIAL_8250_BOCA is not set
# CONFIG_SERIAL_8250_EXAR_ST16C554 is not set
# CONFIG_SERIAL_8250_HUB6 is not setCONFIG_SERIAL_8250_SHARE_IRQ=yCONFIG_SERIAL_8250_DETECT_IRQ=yCONFIG_SERIAL_8250_RSA=y
#
# Non-8250 serial port support
#CONFIG_SERIAL_CORE=yCONFIG_SERIAL_CORE_CONSOLE=yCONFIG_SERIAL_JSM=mCONFIG_UNIX98_PTYS=y
# CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
# CONFIG_LEGACY_PTYS is not setCONFIG_PRINTER=mCONFIG_LP_CONSOLE=yCONFIG_PPDEV=mCONFIG_IPMI_HANDLER=m
# CONFIG_IPMI_PANIC_EVENT is not setCONFIG_IPMI_DEVICE_INTERFACE=mCONFIG_IPMI_SI=mCONFIG_IPMI_WATCHDOG=mCONFIG_IPMI_POWEROFF=mCONFIG_HW_RANDOM=y
# CONFIG_HW_RANDOM_TIMERIOMEM is not setCONFIG_HW_RANDOM_INTEL=mCONFIG_HW_RANDOM_AMD=mCONFIG_HW_RANDOM_GEODE=mCONFIG_HW_RANDOM_VIA=mCONFIG_NVRAM=yCONFIG_RTC=y
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set
#
# PCMCIA character devices
#
# CONFIG_SYNCLINK_CS is not setCONFIG_CARDMAN_4000=mCONFIG_CARDMAN_4040=m
# CONFIG_IPWIRELESS is not setCONFIG_MWAVE=m
# CONFIG_PC8736x_GPIO is not set
# CONFIG_NSC_GPIO is not set
# CONFIG_CS5535_GPIO is not set
# CONFIG_RAW_DRIVER is not setCONFIG_HPET=y
# CONFIG_HPET_MMAP is not setCONFIG_HANGCHECK_TIMER=m
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not setCONFIG_DEVPORT=yCONFIG_I2C=mCONFIG_I2C_BOARDINFO=yCONFIG_I2C_CHARDEV=mCONFIG_I2C_HELPER_AUTO=yCONFIG_I2C_ALGOBIT=mCONFIG_I2C_ALGOPCA=m
#
# I2C Hardware Bus support
#
#
# PC SMBus host controller drivers
#CONFIG_I2C_ALI1535=mCONFIG_I2C_ALI1563=mCONFIG_I2C_ALI15X3=mCONFIG_I2C_AMD756=mCONFIG_I2C_AMD756_S4882=m
# CONFIG_I2C_AMD8111 is not setCONFIG_I2C_I801=m
# CONFIG_I2C_ISCH is not setCONFIG_I2C_PIIX4=mCONFIG_I2C_NFORCE2=m
# CONFIG_I2C_NFORCE2_S4985 is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not setCONFIG_I2C_VIA=mCONFIG_I2C_VIAPRO=m
#
# I2C system bus drivers (mostly embedded / system-on-chip) #
# CONFIG_I2C_OCORES is not setCONFIG_I2C_SIMTEC=m
#
# External I2C/SMBus adapter drivers
#CONFIG_I2C_PARPORT=mCONFIG_I2C_PARPORT_LIGHT=m
# CONFIG_I2C_TAOS_EVM is not set
# CONFIG_I2C_TINY_USB is not set
#
# Graphics adapter I2C/DDC channel drivers
#CONFIG_I2C_VOODOO3=m
#
# Other I2C/SMBus bus drivers
#CONFIG_I2C_PCA_ISA=m
# CONFIG_I2C_PCA_PLATFORM is not setCONFIG_I2C_STUB=m
# CONFIG_SCx200_ACB is not set
#
# Miscellaneous I2C Chip support
#
# CONFIG_DS1682 is not set
# CONFIG_SENSORS_PCF8574 is not set
# CONFIG_PCF8575 is not set
# CONFIG_SENSORS_PCA9539 is not setCONFIG_SENSORS_MAX6875=m
# CONFIG_SENSORS_TSL2550 is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set
# CONFIG_SPI is not setCONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
# CONFIG_GPIOLIB is not set
# CONFIG_W1 is not setCONFIG_POWER_SUPPLY=y
# CONFIG_POWER_SUPPLY_DEBUG is not set
# CONFIG_PDA_POWER is not set
# CONFIG_BATTERY_DS2760 is not set
# CONFIG_BATTERY_BQ27x00 is not setCONFIG_HWMON=mCONFIG_HWMON_VID=m
# CONFIG_SENSORS_ABITUGURU is not set
# CONFIG_SENSORS_ABITUGURU3 is not set
# CONFIG_SENSORS_AD7414 is not setCONFIG_SENSORS_AD7418=m
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ADM1025 is not set
# CONFIG_SENSORS_ADM1026 is not set
# CONFIG_SENSORS_ADM1029 is not set
# CONFIG_SENSORS_ADM1031 is not set
# CONFIG_SENSORS_ADM9240 is not set
# CONFIG_SENSORS_ADT7462 is not set
# CONFIG_SENSORS_ADT7470 is not set
# CONFIG_SENSORS_ADT7473 is not set
# CONFIG_SENSORS_ADT7475 is not set
# CONFIG_SENSORS_K8TEMP is not set
# CONFIG_SENSORS_ASB100 is not set
# CONFIG_SENSORS_ATK0110 is not set
# CONFIG_SENSORS_ATXP1 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_I5K_AMB is not set
# CONFIG_SENSORS_F71805F is not set
# CONFIG_SENSORS_F71882FG is not set
# CONFIG_SENSORS_F75375S is not set
# CONFIG_SENSORS_FSCHER is not set
# CONFIG_SENSORS_FSCPOS is not set
# CONFIG_SENSORS_FSCHMD is not set
# CONFIG_SENSORS_G760A is not set
# CONFIG_SENSORS_GL518SM is not set
# CONFIG_SENSORS_GL520SM is not setCONFIG_SENSORS_CORETEMP=m
# CONFIG_SENSORS_IBMAEM is not set
# CONFIG_SENSORS_IBMPEX is not set
# CONFIG_SENSORS_IT87 is not set
# CONFIG_SENSORS_LM63 is not set
# CONFIG_SENSORS_LM75 is not set
# CONFIG_SENSORS_LM77 is not set
# CONFIG_SENSORS_LM78 is not set
# CONFIG_SENSORS_LM80 is not set
# CONFIG_SENSORS_LM83 is not set
# CONFIG_SENSORS_LM85 is not set
# CONFIG_SENSORS_LM87 is not set
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_LM92 is not set
# CONFIG_SENSORS_LM93 is not set
# CONFIG_SENSORS_LTC4215 is not set
# CONFIG_SENSORS_LTC4245 is not set
# CONFIG_SENSORS_LM95241 is not set
# CONFIG_SENSORS_MAX1619 is not set
# CONFIG_SENSORS_MAX6650 is not set
# CONFIG_SENSORS_PC87360 is not set
# CONFIG_SENSORS_PC87427 is not set
# CONFIG_SENSORS_PCF8591 is not setCONFIG_SENSORS_SIS5595=m
# CONFIG_SENSORS_DME1737 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_SMSC47M192 is not set
# CONFIG_SENSORS_SMSC47B397 is not set
# CONFIG_SENSORS_ADS7828 is not set
# CONFIG_SENSORS_THMC50 is not setCONFIG_SENSORS_VIA686A=mCONFIG_SENSORS_VT1211=mCONFIG_SENSORS_VT8231=m
# CONFIG_SENSORS_W83781D is not set
# CONFIG_SENSORS_W83791D is not set
# CONFIG_SENSORS_W83792D is not set
# CONFIG_SENSORS_W83793 is not set
# CONFIG_SENSORS_W83L785TS is not set
# CONFIG_SENSORS_W83L786NG is not set
# CONFIG_SENSORS_W83627HF is not set
# CONFIG_SENSORS_W83627EHF is not setCONFIG_SENSORS_HDAPS=m
# CONFIG_SENSORS_LIS3LV02D is not set
# CONFIG_SENSORS_APPLESMC is not set
# CONFIG_HWMON_DEBUG_CHIP is not setCONFIG_THERMAL=y
# CONFIG_WATCHDOG is not setCONFIG_SSB_POSSIBLE=y
#
# Sonics Silicon Backplane
#CONFIG_SSB=mCONFIG_SSB_SPROM=yCONFIG_SSB_PCIHOST_POSSIBLE=yCONFIG_SSB_PCIHOST=y
# CONFIG_SSB_B43_PCI_BRIDGE is not setCONFIG_SSB_PCMCIAHOST_POSSIBLE=yCONFIG_SSB_PCMCIAHOST=y
# CONFIG_SSB_DEBUG is not setCONFIG_SSB_DRIVER_PCICORE_POSSIBLE=yCONFIG_SSB_DRIVER_PCICORE=y
#
# Multifunction device drivers
#
# CONFIG_MFD_CORE is not set
# CONFIG_MFD_SM501 is not set
# CONFIG_HTC_PASIC3 is not set
# CONFIG_MFD_TMIO is not set
# CONFIG_MFD_WM8400 is not set
# CONFIG_MFD_WM8350_I2C is not set
# CONFIG_MFD_PCF50633 is not set
# CONFIG_REGULATOR is not set
#
# Multimedia devices
#
#
# Multimedia core support
#CONFIG_VIDEO_DEV=mCONFIG_VIDEO_V4L2_COMMON=mCONFIG_VIDEO_ALLOW_V4L1=yCONFIG_VIDEO_V4L1_COMPAT=y
# CONFIG_DVB_CORE is not setCONFIG_VIDEO_MEDIA=m
#
# Multimedia drivers
#
# CONFIG_MEDIA_ATTACH is not setCONFIG_MEDIA_TUNER=m
# CONFIG_MEDIA_TUNER_CUSTOMISE is not setCONFIG_MEDIA_TUNER_SIMPLE=mCONFIG_MEDIA_TUNER_TDA8290=mCONFIG_MEDIA_TUNER_TDA9887=mCONFIG_MEDIA_TUNER_TEA5761=mCONFIG_MEDIA_TUNER_TEA5767=mCONFIG_MEDIA_TUNER_MT20XX=mCONFIG_MEDIA_TUNER_XC2028=mCONFIG_MEDIA_TUNER_XC5000=mCONFIG_MEDIA_TUNER_MC44S803=mCONFIG_VIDEO_V4L2=mCONFIG_VIDEO_V4L1=mCONFIG_VIDEOBUF_GEN=mCONFIG_VIDEOBUF_DMA_SG=mCONFIG_VIDEO_BTCX=mCONFIG_VIDEO_IR=mCONFIG_VIDEO_TVEEPROM=mCONFIG_VIDEO_TUNER=mCONFIG_VIDEO_CAPTURE_DRIVERS=y
# CONFIG_VIDEO_ADV_DEBUG is not set
# CONFIG_VIDEO_FIXED_MINOR_RANGES is not set
# CONFIG_VIDEO_HELPER_CHIPS_AUTO is not setCONFIG_VIDEO_IR_I2C=m
#
# Encoders/decoders and other helper chips
#
#
# Audio decoders
#CONFIG_VIDEO_TVAUDIO=mCONFIG_VIDEO_TDA7432=mCONFIG_VIDEO_TDA9840=mCONFIG_VIDEO_TDA9875=mCONFIG_VIDEO_TEA6415C=mCONFIG_VIDEO_TEA6420=mCONFIG_VIDEO_MSP3400=m
# CONFIG_VIDEO_CS5345 is not setCONFIG_VIDEO_CS53L32A=mCONFIG_VIDEO_M52790=mCONFIG_VIDEO_TLV320AIC23B=mCONFIG_VIDEO_WM8775=mCONFIG_VIDEO_WM8739=mCONFIG_VIDEO_VP27SMPX=m
#
# RDS decoders
#
# CONFIG_VIDEO_SAA6588 is not set
#
# Video decoders
#CONFIG_VIDEO_BT819=mCONFIG_VIDEO_BT856=mCONFIG_VIDEO_BT866=mCONFIG_VIDEO_KS0127=mCONFIG_VIDEO_OV7670=m
# CONFIG_VIDEO_TCM825X is not setCONFIG_VIDEO_SAA7110=mCONFIG_VIDEO_SAA711X=mCONFIG_VIDEO_SAA717X=mCONFIG_VIDEO_SAA7191=m
# CONFIG_VIDEO_TVP514X is not setCONFIG_VIDEO_TVP5150=mCONFIG_VIDEO_VPX3220=m
#
# Video and audio decoders
#CONFIG_VIDEO_CX25840=m
#
# MPEG video encoders
#CONFIG_VIDEO_CX2341X=m
#
# Video encoders
#CONFIG_VIDEO_SAA7127=mCONFIG_VIDEO_SAA7185=mCONFIG_VIDEO_ADV7170=mCONFIG_VIDEO_ADV7175=m
#
# Video improvement chips
#CONFIG_VIDEO_UPD64031A=mCONFIG_VIDEO_UPD64083=m
# CONFIG_VIDEO_VIVI is not setCONFIG_VIDEO_BT848=m
# CONFIG_VIDEO_PMS is not set
# CONFIG_VIDEO_BWQCAM is not set
# CONFIG_VIDEO_CQCAM is not set
# CONFIG_VIDEO_W9966 is not setCONFIG_VIDEO_CPIA=mCONFIG_VIDEO_CPIA_PP=mCONFIG_VIDEO_CPIA_USB=mCONFIG_VIDEO_CPIA2=m
# CONFIG_VIDEO_SAA5246A is not set
# CONFIG_VIDEO_SAA5249 is not set
# CONFIG_VIDEO_STRADIS is not setCONFIG_VIDEO_ZORAN=m
# CONFIG_VIDEO_ZORAN_DC30 is not setCONFIG_VIDEO_ZORAN_ZR36060=mCONFIG_VIDEO_ZORAN_BUZ=m
# CONFIG_VIDEO_ZORAN_DC10 is not setCONFIG_VIDEO_ZORAN_LML33=m
# CONFIG_VIDEO_ZORAN_LML33R10 is not set
# CONFIG_VIDEO_ZORAN_AVS6EYES is not set
# CONFIG_VIDEO_SAA7134 is not set
# CONFIG_VIDEO_MXB is not set
# CONFIG_VIDEO_HEXIUM_ORION is not set
# CONFIG_VIDEO_HEXIUM_GEMINI is not set
# CONFIG_VIDEO_CX88 is not setCONFIG_VIDEO_IVTV=m
# CONFIG_VIDEO_FB_IVTV is not set
# CONFIG_VIDEO_CAFE_CCIC is not set
# CONFIG_SOC_CAMERA is not set
# CONFIG_V4L_USB_DRIVERS is not setCONFIG_RADIO_ADAPTERS=y
# CONFIG_RADIO_CADET is not set
# CONFIG_RADIO_RTRACK is not set
# CONFIG_RADIO_RTRACK2 is not set
# CONFIG_RADIO_AZTECH is not set
# CONFIG_RADIO_GEMTEK is not set
# CONFIG_RADIO_GEMTEK_PCI is not setCONFIG_RADIO_MAXIRADIO=mCONFIG_RADIO_MAESTRO=m
# CONFIG_RADIO_SF16FMI is not set
# CONFIG_RADIO_SF16FMR2 is not set
# CONFIG_RADIO_TERRATEC is not set
# CONFIG_RADIO_TRUST is not set
# CONFIG_RADIO_TYPHOON is not set
# CONFIG_RADIO_ZOLTRIX is not setCONFIG_USB_DSBR=m
# CONFIG_USB_SI470X is not set
# CONFIG_USB_MR800 is not set
# CONFIG_RADIO_TEA5764 is not setCONFIG_DAB=yCONFIG_USB_DABUSB=m
#
# Graphics support
#CONFIG_AGP=yCONFIG_AGP_ALI=yCONFIG_AGP_ATI=y
# CONFIG_AGP_AMD is not set
# CONFIG_AGP_AMD64 is not setCONFIG_AGP_INTEL=yCONFIG_AGP_NVIDIA=yCONFIG_AGP_SIS=y
# CONFIG_AGP_SWORKS is not setCONFIG_AGP_VIA=yCONFIG_AGP_EFFICEON=yCONFIG_DRM=mCONFIG_DRM_TDFX=mCONFIG_DRM_R128=mCONFIG_DRM_RADEON=mCONFIG_DRM_I810=mCONFIG_DRM_I830=mCONFIG_DRM_I915=m
# CONFIG_DRM_I915_KMS is not set
# CONFIG_DRM_MGA is not setCONFIG_DRM_SIS=m
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not setCONFIG_VGASTATE=mCONFIG_VIDEO_OUTPUT_CONTROL=mCONFIG_FB=y
# CONFIG_FIRMWARE_EDID is not setCONFIG_FB_DDC=mCONFIG_FB_BOOT_VESA_SUPPORT=yCONFIG_FB_CFB_FILLRECT=yCONFIG_FB_CFB_COPYAREA=yCONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
# CONFIG_FB_SYS_FILLRECT is not set
# CONFIG_FB_SYS_COPYAREA is not set
# CONFIG_FB_SYS_IMAGEBLIT is not set
# CONFIG_FB_FOREIGN_ENDIAN is not set
# CONFIG_FB_SYS_FOPS is not setCONFIG_FB_SVGALIB=m
# CONFIG_FB_MACMODES is not setCONFIG_FB_BACKLIGHT=yCONFIG_FB_MODE_HELPERS=yCONFIG_FB_TILEBLITTING=y
#
# Frame buffer hardware drivers
#
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not setCONFIG_FB_VESA=y
# CONFIG_FB_EFI is not set
# CONFIG_FB_N411 is not set
# CONFIG_FB_HGA is not set
# CONFIG_FB_S1D13XXX is not setCONFIG_FB_NVIDIA=mCONFIG_FB_NVIDIA_I2C=y
# CONFIG_FB_NVIDIA_DEBUG is not setCONFIG_FB_NVIDIA_BACKLIGHT=y
# CONFIG_FB_RIVA is not set
# CONFIG_FB_I810 is not set
# CONFIG_FB_LE80578 is not set
# CONFIG_FB_INTEL is not set
# CONFIG_FB_MATROX is not setCONFIG_FB_RADEON=mCONFIG_FB_RADEON_I2C=yCONFIG_FB_RADEON_BACKLIGHT=y
# CONFIG_FB_RADEON_DEBUG is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not setCONFIG_FB_S3=mCONFIG_FB_SAVAGE=mCONFIG_FB_SAVAGE_I2C=yCONFIG_FB_SAVAGE_ACCEL=y
# CONFIG_FB_SIS is not set
# CONFIG_FB_VIA is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_VT8623 is not setCONFIG_FB_TRIDENT=m
# CONFIG_FB_ARK is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_CARMINE is not set
# CONFIG_FB_GEODE is not set
# CONFIG_FB_VIRTUAL is not set
# CONFIG_FB_METRONOME is not set
# CONFIG_FB_MB862XX is not set
# CONFIG_FB_BROADSHEET is not setCONFIG_BACKLIGHT_LCD_SUPPORT=yCONFIG_LCD_CLASS_DEVICE=m
# CONFIG_LCD_ILI9320 is not set
# CONFIG_LCD_PLATFORM is not setCONFIG_BACKLIGHT_CLASS_DEVICE=yCONFIG_BACKLIGHT_GENERIC=yCONFIG_BACKLIGHT_PROGEAR=m
# CONFIG_BACKLIGHT_MBP_NVIDIA is not set
# CONFIG_BACKLIGHT_SAHARA is not set
#
# Display device support
#CONFIG_DISPLAY_SUPPORT=m
#
# Display hardware drivers
#
#
# Console display driver support
#CONFIG_VGA_CONSOLE=yCONFIG_VGACON_SOFT_SCROLLBACK=yCONFIG_VGACON_SOFT_SCROLLBACK_SIZE=64
# CONFIG_MDA_CONSOLE is not setCONFIG_DUMMY_CONSOLE=yCONFIG_FRAMEBUFFER_CONSOLE=yCONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=yCONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=yCONFIG_LOGO=y
# CONFIG_LOGO_LINUX_MONO is not set
# CONFIG_LOGO_LINUX_VGA16 is not setCONFIG_LOGO_LINUX_CLUT224=y
# CONFIG_SOUND is not set
# CONFIG_HID_SUPPORT is not setCONFIG_USB_SUPPORT=yCONFIG_USB_ARCH_HAS_HCD=yCONFIG_USB_ARCH_HAS_OHCI=yCONFIG_USB_ARCH_HAS_EHCI=yCONFIG_USB=y
# CONFIG_USB_DEBUG is not set
# CONFIG_USB_ANNOUNCE_NEW_DEVICES is not set
#
# Miscellaneous USB options
#CONFIG_USB_DEVICEFS=y
# CONFIG_USB_DEVICE_CLASS is not set
# CONFIG_USB_DYNAMIC_MINORS is not setCONFIG_USB_SUSPEND=y
# CONFIG_USB_OTG is not set
# CONFIG_USB_MON is not set
# CONFIG_USB_WUSB is not set
# CONFIG_USB_WUSB_CBAF is not set
#
# USB Host Controller Drivers
#
# CONFIG_USB_C67X00_HCD is not setCONFIG_USB_EHCI_HCD=mCONFIG_USB_EHCI_ROOT_HUB_TT=yCONFIG_USB_EHCI_TT_NEWSCHED=y
# CONFIG_USB_OXU210HP_HCD is not set
# CONFIG_USB_ISP116X_HCD is not set
# CONFIG_USB_ISP1760_HCD is not setCONFIG_USB_OHCI_HCD=m
# CONFIG_USB_OHCI_HCD_SSB is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not setCONFIG_USB_OHCI_LITTLE_ENDIAN=yCONFIG_USB_UHCI_HCD=m
# CONFIG_USB_U132_HCD is not set
# CONFIG_USB_SL811_HCD is not set
# CONFIG_USB_R8A66597_HCD is not set
# CONFIG_USB_WHCI_HCD is not set
# CONFIG_USB_HWA_HCD is not set
#
# USB Device Class drivers
#
# CONFIG_USB_ACM is not set
# CONFIG_USB_PRINTER is not set
# CONFIG_USB_WDM is not set
# CONFIG_USB_TMC is not set
#
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may #
#
# also be needed; see USB_STORAGE Help for more info # CONFIG_USB_STORAGE=m # CONFIG_USB_STORAGE_DEBUG is not set CONFIG_USB_STORAGE_DATAFAB=m CONFIG_USB_STORAGE_FREECOM=m # CONFIG_USB_STORAGE_ISD200 is not set CONFIG_USB_STORAGE_USBAT=m # CONFIG_USB_STORAGE_SDDR09 is not set # CONFIG_USB_STORAGE_SDDR55 is not set # CONFIG_USB_STORAGE_JUMPSHOT is not set # CONFIG_USB_STORAGE_ALAUDA is not set # CONFIG_USB_STORAGE_ONETOUCH is not set # CONFIG_USB_STORAGE_KARMA is not set # CONFIG_USB_STORAGE_CYPRESS_ATACB is not set # CONFIG_USB_LIBUSUAL is not set
#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set
#
# USB port drivers
#
# CONFIG_USB_USS720 is not setCONFIG_USB_SERIAL=mCONFIG_USB_EZUSB=yCONFIG_USB_SERIAL_GENERIC=y
# CONFIG_USB_SERIAL_AIRCABLE is not set
# CONFIG_USB_SERIAL_ARK3116 is not set
# CONFIG_USB_SERIAL_BELKIN is not set
# CONFIG_USB_SERIAL_CH341 is not set
# CONFIG_USB_SERIAL_WHITEHEAT is not set
# CONFIG_USB_SERIAL_DIGI_ACCELEPORT is not set
# CONFIG_USB_SERIAL_CP210X is not set
# CONFIG_USB_SERIAL_CYPRESS_M8 is not setCONFIG_USB_SERIAL_EMPEG=m
# CONFIG_USB_SERIAL_FTDI_SIO is not set
# CONFIG_USB_SERIAL_FUNSOFT is not set
# CONFIG_USB_SERIAL_VISOR is not set
# CONFIG_USB_SERIAL_IPAQ is not set
# CONFIG_USB_SERIAL_IR is not set
# CONFIG_USB_SERIAL_EDGEPORT is not set
# CONFIG_USB_SERIAL_EDGEPORT_TI is not set
# CONFIG_USB_SERIAL_GARMIN is not set
# CONFIG_USB_SERIAL_IPW is not set
# CONFIG_USB_SERIAL_IUU is not set
# CONFIG_USB_SERIAL_KEYSPAN_PDA is not setCONFIG_USB_SERIAL_KEYSPAN=m
# CONFIG_USB_SERIAL_KEYSPAN_MPR is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28 is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28X is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28XA is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28XB is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19 is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA18X is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19W is not setCONFIG_USB_SERIAL_KEYSPAN_USA19QW=yCONFIG_USB_SERIAL_KEYSPAN_USA19QI=yCONFIG_USB_SERIAL_KEYSPAN_USA49W=yCONFIG_USB_SERIAL_KEYSPAN_USA49WLC=y
# CONFIG_USB_SERIAL_KLSI is not set
# CONFIG_USB_SERIAL_KOBIL_SCT is not set
# CONFIG_USB_SERIAL_MCT_U232 is not set
# CONFIG_USB_SERIAL_MOS7720 is not set
# CONFIG_USB_SERIAL_MOS7840 is not set
# CONFIG_USB_SERIAL_MOTOROLA is not set
# CONFIG_USB_SERIAL_NAVMAN is not set
# CONFIG_USB_SERIAL_PL2303 is not set
# CONFIG_USB_SERIAL_OTI6858 is not set
# CONFIG_USB_SERIAL_QUALCOMM is not set
# CONFIG_USB_SERIAL_SPCP8X5 is not set
# CONFIG_USB_SERIAL_HP4X is not set
# CONFIG_USB_SERIAL_SAFE is not set
# CONFIG_USB_SERIAL_SIEMENS_MPI is not set
# CONFIG_USB_SERIAL_SIERRAWIRELESS is not set
# CONFIG_USB_SERIAL_SYMBOL is not set
# CONFIG_USB_SERIAL_TI is not set
# CONFIG_USB_SERIAL_CYBERJACK is not set
# CONFIG_USB_SERIAL_XIRCOM is not set
# CONFIG_USB_SERIAL_OPTION is not set
# CONFIG_USB_SERIAL_OMNINET is not set
# CONFIG_USB_SERIAL_OPTICON is not set
# CONFIG_USB_SERIAL_DEBUG is not set
#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_ADUTUX is not set
# CONFIG_USB_SEVSEG is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_BERRY_CHARGE is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYPRESS_CY7C63 is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_IDMOUSE is not setCONFIG_USB_FTDI_ELAN=m
# CONFIG_USB_APPLEDISPLAY is not set
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_LD is not set
# CONFIG_USB_TRANCEVIBRATOR is not set
# CONFIG_USB_IOWARRIOR is not set
# CONFIG_USB_TEST is not set
# CONFIG_USB_ISIGHTFW is not set
# CONFIG_USB_VST is not set
# CONFIG_USB_GADGET is not set
#
# OTG and related infrastructure
#
# CONFIG_NOP_USB_XCEIV is not set
# CONFIG_UWB is not set
# CONFIG_MMC is not set
# CONFIG_MEMSTICK is not setCONFIG_NEW_LEDS=yCONFIG_LEDS_CLASS=y
#
# LED drivers
#
# CONFIG_LEDS_ALIX2 is not set
# CONFIG_LEDS_PCA9532 is not set
# CONFIG_LEDS_LP5521 is not set
# CONFIG_LEDS_CLEVO_MAIL is not set
# CONFIG_LEDS_PCA955X is not set
# CONFIG_LEDS_BD2802 is not set
#
# LED Triggers
#CONFIG_LEDS_TRIGGERS=yCONFIG_LEDS_TRIGGER_TIMER=m
# CONFIG_LEDS_TRIGGER_HEARTBEAT is not set
# CONFIG_LEDS_TRIGGER_BACKLIGHT is not set
# CONFIG_LEDS_TRIGGER_DEFAULT_ON is not set
#
# iptables trigger is under Netfilter config (LED target) # # CONFIG_ACCESSIBILITY is not set # CONFIG_INFINIBAND is not set # CONFIG_EDAC is not set # CONFIG_RTC_CLASS is not set # CONFIG_DMADEVICES is not set # CONFIG_AUXDISPLAY is not set CONFIG_UIO=m # CONFIG_UIO_CIF is not set # CONFIG_UIO_PDRV is not set # CONFIG_UIO_PDRV_GENIRQ is not set # CONFIG_UIO_SMX is not set # CONFIG_UIO_AEC is not set # CONFIG_UIO_SERCOS3 is not set # CONFIG_STAGING is not set CONFIG_X86_PLATFORM_DEVICES=y # CONFIG_ASUS_LAPTOP is not set # CONFIG_FUJITSU_LAPTOP is not set # CONFIG_TC1100_WMI is not set # CONFIG_MSI_LAPTOP is not set # CONFIG_PANASONIC_LAPTOP is not set # CONFIG_COMPAL_LAPTOP is not set # CONFIG_THINKPAD_ACPI is not set # CONFIG_INTEL_MENLOW is not set # CONFIG_EEEPC_LAPTOP is not set # CONFIG_ACPI_WMI is not set # CONFIG_ACPI_ASUS is not set # CONFIG_ACPI_TOSHIBA is not set
#
# Firmware Drivers
#CONFIG_EDD=m
# CONFIG_EDD_OFF is not setCONFIG_FIRMWARE_MEMMAP=yCONFIG_EFI_VARS=y
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not setCONFIG_DMIID=y
# CONFIG_ISCSI_IBFT_FIND is not set
#
# File systems
#CONFIG_EXT2_FS=m
# CONFIG_EXT2_FS_XATTR is not setCONFIG_EXT2_FS_XIP=yCONFIG_EXT3_FS=m
# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not setCONFIG_EXT3_FS_XATTR=yCONFIG_EXT3_FS_POSIX_ACL=yCONFIG_EXT3_FS_SECURITY=yCONFIG_EXT4_FS=mCONFIG_EXT4DEV_COMPAT=yCONFIG_EXT4_FS_XATTR=yCONFIG_EXT4_FS_POSIX_ACL=yCONFIG_EXT4_FS_SECURITY=yCONFIG_FS_XIP=yCONFIG_JBD=m
# CONFIG_JBD_DEBUG is not setCONFIG_JBD2=m
# CONFIG_JBD2_DEBUG is not setCONFIG_FS_MBCACHE=m
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not setCONFIG_FS_POSIX_ACL=yCONFIG_FILE_LOCKING=y
# CONFIG_XFS_FS is not set
# CONFIG_GFS2_FS is not set
# CONFIG_OCFS2_FS is not set
# CONFIG_BTRFS_FS is not setCONFIG_DNOTIFY=yCONFIG_INOTIFY=yCONFIG_INOTIFY_USER=y
# CONFIG_QUOTA is not set
# CONFIG_AUTOFS_FS is not setCONFIG_AUTOFS4_FS=mCONFIG_FUSE_FS=mCONFIG_GENERIC_ACL=y
#
# Caches
#
# CONFIG_FSCACHE is not set
#
# CD-ROM/DVD Filesystems
#CONFIG_ISO9660_FS=yCONFIG_JOLIET=yCONFIG_ZISOFS=yCONFIG_UDF_FS=yCONFIG_UDF_NLS=y
#
# DOS/FAT/NT Filesystems
#CONFIG_FAT_FS=mCONFIG_MSDOS_FS=mCONFIG_VFAT_FS=mCONFIG_FAT_DEFAULT_CODEPAGE=437CONFIG_FAT_DEFAULT_IOCHARSET="ascii"
# CONFIG_NTFS_FS is not set
#
# Pseudo filesystems
#CONFIG_PROC_FS=yCONFIG_PROC_KCORE=yCONFIG_PROC_VMCORE=yCONFIG_PROC_SYSCTL=yCONFIG_PROC_PAGE_MONITOR=yCONFIG_SYSFS=yCONFIG_TMPFS=yCONFIG_TMPFS_POSIX_ACL=yCONFIG_HUGETLBFS=yCONFIG_HUGETLB_PAGE=yCONFIG_CONFIGFS_FS=mCONFIG_MISC_FILESYSTEMS=y
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not setCONFIG_CRAMFS=m
# CONFIG_SQUASHFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_OMFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not setCONFIG_ROMFS_FS=mCONFIG_ROMFS_BACKED_BY_BLOCK=y
# CONFIG_ROMFS_BACKED_BY_MTD is not set
# CONFIG_ROMFS_BACKED_BY_BOTH is not setCONFIG_ROMFS_ON_BLOCK=y
# CONFIG_SYSV_FS is not setCONFIG_UFS_FS=m
# CONFIG_UFS_FS_WRITE is not set
# CONFIG_UFS_DEBUG is not set
# CONFIG_NILFS2_FS is not setCONFIG_NETWORK_FILESYSTEMS=yCONFIG_NFS_FS=mCONFIG_NFS_V3=yCONFIG_NFS_V3_ACL=yCONFIG_NFS_V4=y
# CONFIG_NFSD is not setCONFIG_LOCKD=mCONFIG_LOCKD_V4=yCONFIG_NFS_ACL_SUPPORT=mCONFIG_NFS_COMMON=yCONFIG_SUNRPC=mCONFIG_SUNRPC_GSS=mCONFIG_RPCSEC_GSS_KRB5=m
# CONFIG_RPCSEC_GSS_SPKM3 is not set
# CONFIG_SMB_FS is not set
# CONFIG_CIFS is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
#
# Partition Types
#CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_OSF_PARTITION is not set
# CONFIG_AMIGA_PARTITION is not set
# CONFIG_ATARI_PARTITION is not set
# CONFIG_MAC_PARTITION is not setCONFIG_MSDOS_PARTITION=yCONFIG_BSD_DISKLABEL=y
# CONFIG_MINIX_SUBPARTITION is not set
# CONFIG_SOLARIS_X86_PARTITION is not set
# CONFIG_UNIXWARE_DISKLABEL is not set
# CONFIG_LDM_PARTITION is not set
# CONFIG_SGI_PARTITION is not set
# CONFIG_ULTRIX_PARTITION is not set
# CONFIG_SUN_PARTITION is not set
# CONFIG_KARMA_PARTITION is not setCONFIG_EFI_PARTITION=y
# CONFIG_SYSV68_PARTITION is not setCONFIG_NLS=yCONFIG_NLS_DEFAULT="utf8"CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not setCONFIG_NLS_CODEPAGE_850=mCONFIG_NLS_CODEPAGE_852=m
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not setCONFIG_NLS_CODEPAGE_863=m
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not setCONFIG_NLS_CODEPAGE_936=mCONFIG_NLS_CODEPAGE_950=mCONFIG_NLS_CODEPAGE_932=m
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not setCONFIG_NLS_ISO8859_8=mCONFIG_NLS_CODEPAGE_1250=mCONFIG_NLS_CODEPAGE_1251=mCONFIG_NLS_ASCII=y
# CONFIG_NLS_ISO8859_1 is not set
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
# CONFIG_NLS_ISO8859_15 is not set
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not setCONFIG_NLS_UTF8=m
# CONFIG_DLM is not set
#
# Kernel hacking
#CONFIG_TRACE_IRQFLAGS_SUPPORT=y
# CONFIG_PRINTK_TIME is not set
# CONFIG_ENABLE_WARN_DEPRECATED is not set
# CONFIG_ENABLE_MUST_CHECK is not setCONFIG_FRAME_WARN=1024CONFIG_MAGIC_SYSRQ=y
# CONFIG_UNUSED_SYMBOLS is not setCONFIG_DEBUG_FS=yCONFIG_HEADERS_CHECK=yCONFIG_DEBUG_KERNEL=yCONFIG_DEBUG_SHIRQ=yCONFIG_DETECT_SOFTLOCKUP=y
# CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC is not setCONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0CONFIG_DETECT_HUNG_TASK=y
# CONFIG_BOOTPARAM_HUNG_TASK_PANIC is not setCONFIG_BOOTPARAM_HUNG_TASK_PANIC_VALUE=0CONFIG_SCHED_DEBUG=yCONFIG_SCHEDSTATS=yCONFIG_TIMER_STATS=y
# CONFIG_DEBUG_OBJECTS is not set
# CONFIG_SLUB_DEBUG_ON is not set
# CONFIG_SLUB_STATS is not setCONFIG_DEBUG_PREEMPT=y
# CONFIG_DEBUG_RT_MUTEXES is not set
# CONFIG_RT_MUTEX_TESTER is not setCONFIG_DEBUG_SPINLOCK=yCONFIG_DEBUG_MUTEXES=yCONFIG_DEBUG_LOCK_ALLOC=yCONFIG_PROVE_LOCKING=yCONFIG_LOCKDEP=y
# CONFIG_LOCK_STAT is not setCONFIG_DEBUG_LOCKDEP=yCONFIG_TRACE_IRQFLAGS=yCONFIG_DEBUG_SPINLOCK_SLEEP=y
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not setCONFIG_STACKTRACE=y
# CONFIG_DEBUG_KOBJECT is not setCONFIG_DEBUG_HIGHMEM=yCONFIG_DEBUG_BUGVERBOSE=yCONFIG_DEBUG_INFO=y
# CONFIG_DEBUG_VM is not set
# CONFIG_DEBUG_VIRTUAL is not set
# CONFIG_DEBUG_WRITECOUNT is not setCONFIG_DEBUG_MEMORY_INIT=yCONFIG_DEBUG_LIST=y
# CONFIG_DEBUG_SG is not set
# CONFIG_DEBUG_NOTIFIERS is not setCONFIG_ARCH_WANT_FRAME_POINTERS=yCONFIG_FRAME_POINTER=y
# CONFIG_BOOT_PRINTK_DELAY is not set
# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_KPROBES_SANITY_TEST is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
# CONFIG_LKDTM is not set
# CONFIG_FAULT_INJECTION is not set
# CONFIG_LATENCYTOP is not setCONFIG_SYSCTL_SYSCALL_CHECK=y
# CONFIG_DEBUG_PAGEALLOC is not setCONFIG_USER_STACKTRACE_SUPPORT=yCONFIG_NOP_TRACER=yCONFIG_HAVE_FTRACE_NMI_ENTER=yCONFIG_HAVE_FUNCTION_TRACER=yCONFIG_HAVE_FUNCTION_GRAPH_TRACER=yCONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=yCONFIG_HAVE_DYNAMIC_FTRACE=yCONFIG_HAVE_FTRACE_MCOUNT_RECORD=yCONFIG_HAVE_FTRACE_SYSCALLS=yCONFIG_TRACER_MAX_TRACE=yCONFIG_RING_BUFFER=yCONFIG_FTRACE_NMI_ENTER=yCONFIG_TRACING=yCONFIG_TRACING_SUPPORT=y
#
# Tracers
#CONFIG_FUNCTION_TRACER=yCONFIG_FUNCTION_GRAPH_TRACER=yCONFIG_IRQSOFF_TRACER=yCONFIG_PREEMPT_TRACER=yCONFIG_SYSPROF_TRACER=yCONFIG_SCHED_TRACER=yCONFIG_CONTEXT_SWITCH_TRACER=yCONFIG_EVENT_TRACER=yCONFIG_FTRACE_SYSCALLS=yCONFIG_BOOT_TRACER=y
# CONFIG_TRACE_BRANCH_PROFILING is not setCONFIG_POWER_TRACER=yCONFIG_STACK_TRACER=y
# CONFIG_KMEMTRACE is not setCONFIG_WORKQUEUE_TRACER=yCONFIG_BLK_DEV_IO_TRACE=yCONFIG_DYNAMIC_FTRACE=yCONFIG_FTRACE_MCOUNT_RECORD=y
# CONFIG_FTRACE_STARTUP_TEST is not setCONFIG_MMIOTRACE=yCONFIG_MMIOTRACE_TEST=m
# CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
# CONFIG_FIREWIRE_OHCI_REMOTE_DMA is not set
# CONFIG_BUILD_DOCSRC is not set
# CONFIG_DYNAMIC_DEBUG is not set
# CONFIG_DMA_API_DEBUG is not setCONFIG_SAMPLES=y
# CONFIG_SAMPLE_MARKERS is not set
# CONFIG_SAMPLE_TRACEPOINTS is not setCONFIG_SAMPLE_KOBJECT=mCONFIG_SAMPLE_KPROBES=mCONFIG_SAMPLE_KRETPROBES=mCONFIG_HAVE_ARCH_KGDB=y
# CONFIG_KGDB is not set
# CONFIG_STRICT_DEVMEM is not setCONFIG_X86_VERBOSE_BOOTUP=yCONFIG_EARLY_PRINTK=y
# CONFIG_EARLY_PRINTK_DBGP is not set
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_PER_CPU_MAPS is not set
# CONFIG_X86_PTDUMP is not setCONFIG_DEBUG_RODATA=y
# CONFIG_DEBUG_RODATA_TEST is not set
# CONFIG_DEBUG_NX_TEST is not setCONFIG_4KSTACKS=yCONFIG_DOUBLEFAULT=yCONFIG_HAVE_MMIOTRACE_SUPPORT=yCONFIG_IO_DELAY_TYPE_0X80=0CONFIG_IO_DELAY_TYPE_0XED=1CONFIG_IO_DELAY_TYPE_UDELAY=2CONFIG_IO_DELAY_TYPE_NONE=3CONFIG_IO_DELAY_0X80=y
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not setCONFIG_DEFAULT_IO_DELAY_TYPE=0
# CONFIG_DEBUG_BOOT_PARAMS is not set
# CONFIG_CPA_DEBUG is not set
# CONFIG_OPTIMIZE_INLINING is not set
#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY is not set
# CONFIG_SECURITYFS is not set
# CONFIG_SECURITY_FILE_CAPABILITIES is not set
# CONFIG_IMA is not setCONFIG_CRYPTO=y
#
# Crypto core or helper
#
# CONFIG_CRYPTO_FIPS is not setCONFIG_CRYPTO_ALGAPI=yCONFIG_CRYPTO_ALGAPI2=yCONFIG_CRYPTO_AEAD2=yCONFIG_CRYPTO_BLKCIPHER=mCONFIG_CRYPTO_BLKCIPHER2=yCONFIG_CRYPTO_HASH=yCONFIG_CRYPTO_HASH2=yCONFIG_CRYPTO_RNG2=yCONFIG_CRYPTO_PCOMP=yCONFIG_CRYPTO_MANAGER=yCONFIG_CRYPTO_MANAGER2=y
# CONFIG_CRYPTO_GF128MUL is not setCONFIG_CRYPTO_NULL=mCONFIG_CRYPTO_WORKQUEUE=y
# CONFIG_CRYPTO_CRYPTD is not set
# CONFIG_CRYPTO_AUTHENC is not set
# CONFIG_CRYPTO_TEST is not set
#
# Authenticated Encryption with Associated Data
#
# CONFIG_CRYPTO_CCM is not set
# CONFIG_CRYPTO_GCM is not set
# CONFIG_CRYPTO_SEQIV is not set
#
# Block modes
#CONFIG_CRYPTO_CBC=m
# CONFIG_CRYPTO_CTR is not set
# CONFIG_CRYPTO_CTS is not set
# CONFIG_CRYPTO_ECB is not set
# CONFIG_CRYPTO_LRW is not set
# CONFIG_CRYPTO_PCBC is not set
# CONFIG_CRYPTO_XTS is not set
#
# Hash modes
#
# CONFIG_CRYPTO_HMAC is not set
# CONFIG_CRYPTO_XCBC is not set
#
# Digest
#CONFIG_CRYPTO_CRC32C=y
# CONFIG_CRYPTO_CRC32C_INTEL is not setCONFIG_CRYPTO_MD4=mCONFIG_CRYPTO_MD5=y
# CONFIG_CRYPTO_MICHAEL_MIC is not set
# CONFIG_CRYPTO_RMD128 is not set
# CONFIG_CRYPTO_RMD160 is not set
# CONFIG_CRYPTO_RMD256 is not set
# CONFIG_CRYPTO_RMD320 is not setCONFIG_CRYPTO_SHA1=yCONFIG_CRYPTO_SHA256=m
# CONFIG_CRYPTO_SHA512 is not set
# CONFIG_CRYPTO_TGR192 is not set
# CONFIG_CRYPTO_WP512 is not set
#
# Ciphers
#CONFIG_CRYPTO_AES=m
# CONFIG_CRYPTO_AES_586 is not set
# CONFIG_CRYPTO_ANUBIS is not set
# CONFIG_CRYPTO_ARC4 is not set
# CONFIG_CRYPTO_BLOWFISH is not set
# CONFIG_CRYPTO_CAMELLIA is not set
# CONFIG_CRYPTO_CAST5 is not set
# CONFIG_CRYPTO_CAST6 is not setCONFIG_CRYPTO_DES=m
# CONFIG_CRYPTO_FCRYPT is not set
# CONFIG_CRYPTO_KHAZAD is not set
# CONFIG_CRYPTO_SALSA20 is not set
# CONFIG_CRYPTO_SALSA20_586 is not set
# CONFIG_CRYPTO_SEED is not set
# CONFIG_CRYPTO_SERPENT is not set
# CONFIG_CRYPTO_TEA is not set
# CONFIG_CRYPTO_TWOFISH is not set
# CONFIG_CRYPTO_TWOFISH_586 is not set
#
# Compression
#
# CONFIG_CRYPTO_DEFLATE is not set
# CONFIG_CRYPTO_ZLIB is not set
# CONFIG_CRYPTO_LZO is not set
#
# Random Number Generation
#
# CONFIG_CRYPTO_ANSI_CPRNG is not set
# CONFIG_CRYPTO_HW is not setCONFIG_HAVE_KVM=yCONFIG_HAVE_KVM_IRQCHIP=y
# CONFIG_VIRTUALIZATION is not setCONFIG_BINARY_PRINTF=y
#
# Library routines
#CONFIG_BITREVERSE=yCONFIG_GENERIC_FIND_FIRST_BIT=yCONFIG_GENERIC_FIND_NEXT_BIT=yCONFIG_GENERIC_FIND_LAST_BIT=yCONFIG_CRC_CCITT=mCONFIG_CRC16=m
# CONFIG_CRC_T10DIF is not setCONFIG_CRC_ITU_T=yCONFIG_CRC32=y
# CONFIG_CRC7 is not set
# CONFIG_LIBCRC32C is not setCONFIG_ZLIB_INFLATE=yCONFIG_ZLIB_DEFLATE=mCONFIG_DECOMPRESS_GZIP=yCONFIG_DECOMPRESS_BZIP2=yCONFIG_DECOMPRESS_LZMA=yCONFIG_HAS_IOMEM=yCONFIG_HAS_IOPORT=yCONFIG_HAS_DMA=yCONFIG_NLATTR=y
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.30-rc4-io (root-bi+AKbBUZKY6gyzm1THtWbp2dZbC/[email protected]) (gcc version 4.1.2 20070925 (Red Hat 4.1.2-33)) #6 SMP PREEMPT Thu May 7 11:07:49 CST 2009 KERNEL supported cpus:
Intel GenuineIntel
AMD AuthenticAMD
NSC Geode by NSC
Cyrix CyrixInstead
Centaur CentaurHauls
Transmeta GenuineTMx86
Transmeta TransmetaCPU
UMC UMC UMC UMC
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009f400 (usable)
BIOS-e820: 000000000009f400 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000003bff0000 (usable)
BIOS-e820: 000000003bff0000 - 000000003bff3000 (ACPI NVS)
BIOS-e820: 000000003bff3000 - 000000003c000000 (ACPI data)
BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
DMI 2.3 present.
Phoenix BIOS detected: BIOS may corrupt low RAM, working around it.
e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)last_pfn = 0x3bff0 max_arch_pfn = 0x100000
MTRR default type: uncachable
MTRR fixed ranges enabled:
00000-9FFFF write-back
A0000-BFFFF uncachable
C0000-C7FFF write-protect
C8000-FFFFF uncachable
MTRR variable ranges enabled:
0 base 000000000 mask FC0000000 write-back
1 base 03C000000 mask FFC000000 uncachable
2 base 0D0000000 mask FF8000000 write-combining
3 disabled
4 disabled
5 disabled
6 disabled
7 disabledinit_memory_mapping: 0000000000000000-00000000377fe000
0000000000 - 0000400000 page 4k
0000400000 - 0037400000 page 2M
0037400000 - 00377fe000 page 4k
kernel direct mapping tables up to 377fe000 @ 10000-15000
RAMDISK: 37d0d000 - 37fefd69
Allocated new RAMDISK: 00100000 - 003e2d69
Move RAMDISK from 0000000037d0d000 - 0000000037fefd68 to 00100000 - 003e2d68
ACPI: RSDP 000f7560 00014 (v00 AWARD )
ACPI: RSDT 3bff3040 0002C (v01 AWARD AWRDACPI 42302E31 AWRD 00000000)
ACPI: FACP 3bff30c0 00074 (v01 AWARD AWRDACPI 42302E31 AWRD 00000000)
ACPI: DSDT 3bff3180 03ABC (v01 AWARD AWRDACPI 00001000 MSFT 0100000E)
ACPI: FACS 3bff0000 00040
ACPI: APIC 3bff6c80 00084 (v01 AWARD AWRDACPI 42302E31 AWRD 00000000)
ACPI: Local APIC address 0xfee00000
71MB HIGHMEM available.
887MB LOWMEM available.
mapped low ram: 0 - 377fe000
low ram: 0 - 377fe000
node 0 low ram: 00000000 - 377fe000
node 0 bootmap 00011000 - 00017f00
(9 early reservations) ==> bootmem [0000000000 - 00377fe000]
#0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000]
#1 [0000001000 - 0000002000] EX TRAMPOLINE ==> [0000001000 - 0000002000]
#2 [0000006000 - 0000007000] TRAMPOLINE ==> [0000006000 - 0000007000]
#3 [0000400000 - 0000c6bd1c] TEXT DATA BSS ==> [0000400000 - 0000c6bd1c]
#4 [000009f400 - 0000100000] BIOS reserved ==> [000009f400 - 0000100000]
#5 [0000c6c000 - 0000c700ed] BRK ==> [0000c6c000 - 0000c700ed]
#6 [0000010000 - 0000011000] PGTABLE ==> [0000010000 - 0000011000]
#7 [0000100000 - 00003e2d69] NEW RAMDISK ==> [0000100000 - 00003e2d69]
#8 [0000011000 - 0000018000] BOOTMAP ==> [0000011000 - 0000018000]
found SMP MP-table at [c00f5ad0] f5ad0
Zone PFN ranges:
DMA 0x00000010 -> 0x00001000
Normal 0x00001000 -> 0x000377fe
HighMem 0x000377fe -> 0x0003bff0
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
0: 0x00000010 -> 0x0000009f
0: 0x00000100 -> 0x0003bff0On node 0 totalpages: 245631free_area_init_node: node 0, pgdat c0778f80, node_mem_map c1000340
DMA zone: 52 pages used for memmap
DMA zone: 0 pages reserved
DMA zone: 3931 pages, LIFO batch:0
Normal zone: 2834 pages used for memmap
Normal zone: 220396 pages, LIFO batch:31
HighMem zone: 234 pages used for memmap
HighMem zone: 18184 pages, LIFO batch:3
Using APIC driver default
ACPI: PM-Timer IO Port: 0x1008
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] disabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] disabled)
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0]) IOAPIC[0]: apic_id 4, version 17, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 dfl dfl)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Enabling APIC mode: Flat. Using 1 I/O APICs
Using ACPI (MADT) for SMP configuration information
SMP: Allowing 4 CPUs, 2 hotplug CPUsnr_irqs_gsi: 24
Allocating PCI resources starting at 40000000 (gap: 3c000000:c2c00000) NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:4 nr_node_ids:1
PERCPU: Embedded 13 pages at c1c3b000, static data 32756 bytes Built 1 zonelists in Zone order, mobility grouping on. Total pages: 242511 Kernel command line: ro root=LABEL=/ rhgb quiet
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done. Initializing CPU#0
Preemptible RCU implementation.
NR_IRQS:512
CPU 0 irqstacks, hard=c1c3b000 soft=c1c3c000
PID hash table entries: 4096 (order: 12, 16384 bytes)
Fast TSC calibration using PIT
Detected 2800.222 MHz processor.
Console: colour VGA+ 80x25
console [tty0] enabled
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
... MAX_LOCKDEP_SUBCLASSES: 8 ... MAX_LOCK_DEPTH: 48 ... MAX_LOCKDEP_KEYS: 8191 ... CLASSHASH_SIZE: 4096 ... MAX_LOCKDEP_ENTRIES: 8192 ... MAX_LOCKDEP_CHAINS: 16384 ... CHAINHASH_SIZE: 8192
memory used by lock dependency info: 2847 kB
per task-struct memory footprint: 1152 bytes
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
allocated 4914560 bytes of page_cgroup
please try cgroup_disable=memory,blkio option if you don't want
Initializing HighMem for node 0 (000377fe:0003bff0)
Memory: 952284k/982976k available (2258k kernel code, 30016k reserved, 1424k data, 320k init, 73672k highmem) virtual kernel memory layout:
fixmap : 0xffedf000 - 0xfffff000 (1152 kB)
pkmap : 0xff800000 - 0xffc00000 (4096 kB)
vmalloc : 0xf7ffe000 - 0xff7fe000 ( 120 MB)
lowmem : 0xc0000000 - 0xf77fe000 ( 887 MB)
.init : 0xc079d000 - 0xc07ed000 ( 320 kB)
.data : 0xc06349ab - 0xc0798cb8 (1424 kB)
.text : 0xc0400000 - 0xc06349ab (2258 kB)Checking if this processor honours the WP bit even in supervisor mode...Ok.
SLUB: Genslabs=13, HWalign=128, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
Calibrating delay loop (skipped), value calculated using timer frequency.. 5600.44 BogoMIPS (lpj=2800222) Mount-cache hash table entries: 512
Initializing cgroup subsys debug
Initializing cgroup subsys ns
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys blkio
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys io
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 1024K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU0: Intel P4/Xeon Extended MCE MSRs (24) available
using mwait in idle threads.
Checking 'hlt' instruction... OK.
ACPI: Core revision 20090320
ftrace: converting mcount calls to 0f 1f 44 00 00
ftrace: allocating 12136 entries in 24 pages
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: Intel(R) Pentium(R) D CPU 2.80GHz stepping 04
lockdep: fixing up alternatives.
CPU 1 irqstacks, hard=c1c4b000 soft=c1c4c000
Booting processor 1 APIC 0x1 ip 0x6000
Initializing CPU#1
Calibrating delay using timer specific routine.. 5599.23 BogoMIPS (lpj=2799617)
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 1024K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 1
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#1.
CPU1: Intel P4/Xeon Extended MCE MSRs (24) available
CPU1: Intel(R) Pentium(R) D CPU 2.80GHz stepping 04
checking TSC synchronization [CPU#0 -> CPU#1]: passed.
Brought up 2 CPUs
Total of 2 processors activated (11199.67 BogoMIPS).
CPU0 attaching sched-domain:
domain 0: span 0-1 level CPU
groups: 0 1
CPU1 attaching sched-domain:
domain 0: span 0-1 level CPU
groups: 1 0net_namespace: 436 bytes
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xfbda0, last bus=1
PCI: Using configuration type 1 for base access
mtrr: your CPUs had inconsistent fixed MTRR settings
mtrr: probably your BIOS does not setup all CPUs.
mtrr: corrected configuration.
bio: create slab <bio-0> at 0
ACPI: EC: Look up EC in DSDT
ACPI: Interpreter enabled
ACPI: (supports S0 S3 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI: No dock devices found.
ACPI: PCI Root Bridge [PCI0] (0000:00)
pci 0000:00:00.0: reg 10 32bit mmio: [0xd0000000-0xd7ffffff]
pci 0000:00:02.5: reg 10 io port: [0x1f0-0x1f7]
pci 0000:00:02.5: reg 14 io port: [0x3f4-0x3f7]
pci 0000:00:02.5: reg 18 io port: [0x170-0x177]
pci 0000:00:02.5: reg 1c io port: [0x374-0x377]
pci 0000:00:02.5: reg 20 io port: [0x4000-0x400f]
pci 0000:00:02.5: PME# supported from D3cold
pci 0000:00:02.5: PME# disabled
pci 0000:00:02.7: reg 10 io port: [0xd000-0xd0ff]
pci 0000:00:02.7: reg 14 io port: [0xd400-0xd47f]
pci 0000:00:02.7: supports D1 D2
pci 0000:00:02.7: PME# supported from D3hot D3cold
pci 0000:00:02.7: PME# disabled
pci 0000:00:03.0: reg 10 32bit mmio: [0xe1104000-0xe1104fff]
pci 0000:00:03.1: reg 10 32bit mmio: [0xe1100000-0xe1100fff]
pci 0000:00:03.2: reg 10 32bit mmio: [0xe1101000-0xe1101fff]
pci 0000:00:03.3: reg 10 32bit mmio: [0xe1102000-0xe1102fff]
pci 0000:00:03.3: PME# supported from D0 D3hot D3cold
pci 0000:00:03.3: PME# disabled
pci 0000:00:05.0: reg 10 io port: [0xd800-0xd807]
pci 0000:00:05.0: reg 14 io port: [0xdc00-0xdc03]
pci 0000:00:05.0: reg 18 io port: [0xe000-0xe007]
pci 0000:00:05.0: reg 1c io port: [0xe400-0xe403]
pci 0000:00:05.0: reg 20 io port: [0xe800-0xe80f]
pci 0000:00:05.0: PME# supported from D3cold
pci 0000:00:05.0: PME# disabled
pci 0000:00:0e.0: reg 10 io port: [0xec00-0xecff]
pci 0000:00:0e.0: reg 14 32bit mmio: [0xe1103000-0xe11030ff]
pci 0000:00:0e.0: reg 30 32bit mmio: [0x000000-0x01ffff]
pci 0000:00:0e.0: supports D1 D2
pci 0000:00:0e.0: PME# supported from D1 D2 D3hot D3cold
pci 0000:00:0e.0: PME# disabled
pci 0000:01:00.0: reg 10 32bit mmio: [0xd8000000-0xdfffffff]
pci 0000:01:00.0: reg 14 32bit mmio: [0xe1000000-0xe101ffff]
pci 0000:01:00.0: reg 18 io port: [0xc000-0xc07f]
pci 0000:01:00.0: supports D1 D2
pci 0000:00:01.0: bridge io port: [0xc000-0xcfff]
pci 0000:00:01.0: bridge 32bit mmio: [0xe1000000-0xe10fffff]
pci 0000:00:01.0: bridge 32bit mmio pref: [0xd8000000-0xdfffffff]pci_bus 0000:00: on NUMA node 0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 10 11 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 10 *11 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 *10 11 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 9 10 11 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 9 10 *11 14 15)
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 *6 7 9 10 11 14 15)
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 *9 10 11 14 15)
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 *5 6 7 9 10 11 14 15)
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 12 devices
ACPI: ACPI bus type pnp unregistered
system 00:00: iomem range 0xc8000-0xcbfff has been reserved
system 00:00: iomem range 0xf0000-0xf7fff could not be reserved
system 00:00: iomem range 0xf8000-0xfbfff could not be reserved
system 00:00: iomem range 0xfc000-0xfffff could not be reserved
system 00:00: iomem range 0x3bff0000-0x3bffffff could not be reserved
system 00:00: iomem range 0xffff0000-0xffffffff has been reserved
system 00:00: iomem range 0x0-0x9ffff could not be reserved
system 00:00: iomem range 0x100000-0x3bfeffff could not be reserved
system 00:00: iomem range 0xffee0000-0xffefffff has been reserved
system 00:00: iomem range 0xfffe0000-0xfffeffff has been reserved
system 00:00: iomem range 0xfec00000-0xfecfffff has been reserved
system 00:00: iomem range 0xfee00000-0xfeefffff has been reserved
system 00:02: ioport range 0x4d0-0x4d1 has been reserved
system 00:02: ioport range 0x800-0x805 has been reserved
system 00:02: ioport range 0x290-0x297 has been reserved
system 00:02: ioport range 0x880-0x88f has been reserved
pci 0000:00:01.0: PCI bridge, secondary bus 0000:01
pci 0000:00:01.0: IO window: 0xc000-0xcfff pci 0000:00:01.0: MEM window: 0xe1000000-0xe10fffff pci 0000:00:01.0: PREFETCH window: 0x000000d8000000-0x000000dfffffff
pci_bus 0000:00: resource 0 io: [0x00-0xffff]pci_bus 0000:00: resource 1 mem: [0x000000-0xffffffff]pci_bus 0000:01: resource 0 io: [0xc000-0xcfff]pci_bus 0000:01: resource 1 mem: [0xe1000000-0xe10fffff]pci_bus 0000:01: resource 2 pref mem [0xd8000000-0xdfffffff]
NET: Registered protocol family 2
IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
TCP bind hash table entries: 65536 (order: 9, 2097152 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
NET: Registered protocol family 1
checking if image is initramfs...
rootfs image is initramfs; unpacking...
Freeing initrd memory: 2955k freed
apm: BIOS version 1.2 Flags 0x07 (Driver version 1.16ac)
apm: disabled - APM is not SMP safe.
highmem bounce pool size: 64 pages
HugeTLB registered 4 MB page size, pre-allocated 0 pages
msgmni has been set to 1722
alg: No test for stdrng (krng)
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254)
io scheduler noop registered
io scheduler cfq registered (default)
pci 0000:01:00.0: Boot video devicepci_hotplug: PCI Hot Plug PCI Core version: 0.5
fan PNP0C0B:00: registered as cooling_device0
ACPI: Fan [FAN] (on)
processor ACPI_CPU:00: registered as cooling_device1
processor ACPI_CPU:01: registered as cooling_device2
thermal LNXTHERM:01: registered as thermal_zone0
ACPI: Thermal Zone [THRM] (62 C)
isapnp: Scanning for PnP cards...
Switched to high resolution mode on CPU 1
Switched to high resolution mode on CPU 0
isapnp: No Plug & Play device found
Real Time Clock Driver v1.12b
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
agpgart-sis 0000:00:00.0: SiS chipset [1039/0661]
agpgart-sis 0000:00:00.0: AGP aperture is 128M @ 0xd0000000
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:07: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:08: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
brd: module loaded
PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x60,0x64 irq 1,12
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
cpuidle: using governor ladder
cpuidle: using governor menu
TCP cubic registered
NET: Registered protocol family 17
Using IPI No-Shortcut mode
registered taskstats version 1
Freeing unused kernel memory: 320k freed
Write protecting the kernel text: 2260k
Write protecting the kernel read-only data: 1120kehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driverehci_hcd 0000:00:03.3: PCI INT D -> GSI 23 (level, low) -> IRQ 23 ehci_hcd 0000:00:03.3: EHCI Host Controllerehci_hcd 0000:00:03.3: new USB bus registered, assigned bus number 1 ehci_hcd 0000:00:03.3: cache line size of 128 is not supportedehci_hcd 0000:00:03.3: irq 23, io mem 0xe1102000ehci_hcd 0000:00:03.3: USB 2.0 started, EHCI 1.00
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 8 ports detectedohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driverohci_hcd 0000:00:03.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20 ohci_hcd 0000:00:03.0: OHCI Host Controllerohci_hcd 0000:00:03.0: new USB bus registered, assigned bus number 2 ohci_hcd 0000:00:03.0: irq 20, io mem 0xe1104000
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 3 ports detectedohci_hcd 0000:00:03.1: PCI INT B -> GSI 21 (level, low) -> IRQ 21 ohci_hcd 0000:00:03.1: OHCI Host Controllerohci_hcd 0000:00:03.1: new USB bus registered, assigned bus number 3 ohci_hcd 0000:00:03.1: irq 21, io mem 0xe1100000
usb usb3: configuration #1 chosen from 1 choice
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 3 ports detectedohci_hcd 0000:00:03.2: PCI INT C -> GSI 22 (level, low) -> IRQ 22 ohci_hcd 0000:00:03.2: OHCI Host Controllerohci_hcd 0000:00:03.2: new USB bus registered, assigned bus number 4 ohci_hcd 0000:00:03.2: irq 22, io mem 0xe1101000
usb usb4: configuration #1 chosen from 1 choice
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 2 ports detecteduhci_hcd: USB Universal Host Controller Interface driver
SCSI subsystem initialized
Driver 'sd' needs updating - please use bus_type methods
libata version 3.00 loaded.pata_sis 0000:00:02.5: version 0.5.2pata_sis 0000:00:02.5: PCI INT A -> GSI 16 (level, low) -> IRQ 16
scsi0 : pata_sis
scsi1 : pata_sis
ata1: PATA max UDMA/133 cmd 0x1f0 ctl 0x3f6 bmdma 0x4000 irq 14
ata2: PATA max UDMA/133 cmd 0x170 ctl 0x376 bmdma 0x4008 irq 15
input: ImPS/2 Logitech Wheel Mouse as /class/input/input0
input: AT Translated Set 2 keyboard as /class/input/input1sata_sis 0000:00:05.0: version 1.0sata_sis 0000:00:05.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17 sata_sis 0000:00:05.0: Detected SiS 180/181/964 chipset in SATA mode scsi2 : sata_sis
scsi3 : sata_sis
ata3: SATA max UDMA/133 cmd 0xd800 ctl 0xdc00 bmdma 0xe800 irq 17
ata4: SATA max UDMA/133 cmd 0xe000 ctl 0xe400 bmdma 0xe808 irq 17
ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata3.00: ATA-7: ST3808110AS, 3.AAE, max UDMA/133
ata3.00: 156301488 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata3.00: configured for UDMA/133
scsi 2:0:0:0: Direct-Access ATA ST3808110AS 3.AA PQ: 0 ANSI: 5
sd 2:0:0:0: [sda] 156301488 512-byte hardware sectors: (80.0 GB/74.5 GiB) sd 2:0:0:0: [sda] Write Protect is off
sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sda: sda1 sda2 < sda5 sda6 sda7 sda8 sda9 >
sd 2:0:0:0: [sda] Attached SCSI disk
ata4: SATA link down (SStatus 0 SControl 300)
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: sda8: orphan cleanup on readonly fsext3_orphan_cleanup: deleting unreferenced inode 3725366ext3_orphan_cleanup: deleting unreferenced inode 3725365ext3_orphan_cleanup: deleting unreferenced inode 3725364
EXT3-fs: sda8: 3 orphan inodes deleted
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with writeback data mode.
r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
r8169 0000:00:0e.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
r8169 0000:00:0e.0: no PCI Express capability
eth0: RTL8110s at 0xf8236000, 00:16:ec:2e:b7:e0, XID 04000000 IRQ 18
sd 2:0:0:0: Attached scsi generic sg0 type 0parport_pc 00:09: reported by Plug and Play ACPI
parport0: PC-style at 0x378 (0x778), irq 7 [PCSPP,TRISTATE]
input: Power Button as /class/input/input2
ACPI: Power Button [PWRF]
input: Power Button as /class/input/input3
ACPI: Power Button [PWRB]
input: Sleep Button as /class/input/input4
ACPI: Sleep Button [FUTS]
ramfs: bad mount option: maxsize=512
EXT3 FS on sda8, internal journal
kjournald starting. Commit interval 5 seconds
EXT3 FS on sda7, internal journal
EXT3-fs: mounted filesystem with writeback data mode.
Adding 1052216k swap on /dev/sda6. Priority:-1 extents:1 across:1052216k
warning: process `kudzu' used the deprecated sysctl system call with 1.23. kudzu[1133] general protection ip:8056968 sp:bffe9e90 error:0
r8169: eth0: link up
r8169: eth0: link up
warning: `dbus-daemon' uses 32-bit capabilities (legacy support in use) CPU0 attaching NULL sched-domain.
CPU1 attaching NULL sched-domain.
CPU0 attaching sched-domain:
domain 0: span 0-1 level CPU
groups: 0 1
CPU1 attaching sched-domain:
domain 0: span 0-1 level CPU
groups: 1 0
=========================================================
[ INFO: possible irq lock inversion dependency detected ] 2.6.30-rc4-io #6
---------------------------------------------------------
rmdir/2186 just changed the state of lock:
(&iocg->lock){+.+...}, at: [<c0513b18>] iocg_destroy+0x2a/0x118
but this lock was taken by another, SOFTIRQ-safe lock in the past:
(&q->__queue_lock){..-...}
and interrupts could create inverse lock ordering between them.
other info that might help us debug this:
3 locks held by rmdir/2186:
#0: (&sb->s_type->i_mutex_key#10/1){+.+.+.}, at: [<c04ae1e8>] do_rmdir+0x5c/0xc8
#1: (cgroup_mutex){+.+.+.}, at: [<c045a15b>] cgroup_diput+0x3c/0xa7
#2: (&iocg->lock){+.+...}, at: [<c0513b18>] iocg_destroy+0x2a/0x118the first lock's dependencies:
-> (&iocg->lock){+.+...} ops: 3 {
HARDIRQ-ON-W at:
[<c044b840>] mark_held_locks+0x3d/0x58
[<c044b963>] trace_hardirqs_on_caller+0x108/0x14c
[<c044b9b2>] trace_hardirqs_on+0xb/0xd
[<c0630883>] _spin_unlock_irq+0x27/0x47
[<c0513baa>] iocg_destroy+0xbc/0x118
[<c045a16a>] cgroup_diput+0x4b/0xa7
[<c04b1dbb>] dentry_iput+0x78/0x9c
[<c04b1e82>] d_kill+0x21/0x3b
[<c04b2f2a>] dput+0xf3/0xfc
[<c04ae226>] do_rmdir+0x9a/0xc8
[<c04ae29d>] sys_rmdir+0x15/0x17
[<c0402a68>] sysenter_do_call+0x12/0x36
[<ffffffff>] 0xffffffff
SOFTIRQ-ON-W at:
[<c044b840>] mark_held_locks+0x3d/0x58
[<c044b97c>] trace_hardirqs_on_caller+0x121/0x14c
[<c044b9b2>] trace_hardirqs_on+0xb/0xd
[<c0630883>] _spin_unlock_irq+0x27/0x47
[<c0513baa>] iocg_destroy+0xbc/0x118
[<c045a16a>] cgroup_diput+0x4b/0xa7
[<c04b1dbb>] dentry_iput+0x78/0x9c
[<c04b1e82>] d_kill+0x21/0x3b
[<c04b2f2a>] dput+0xf3/0xfc
[<c04ae226>] do_rmdir+0x9a/0xc8
[<c04ae29d>] sys_rmdir+0x15/0x17
[<c0402a68>] sysenter_do_call+0x12/0x36
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c06304ea>] _spin_lock_irq+0x30/0x3f
[<c05119bd>] io_alloc_root_group+0x104/0x155
[<c05133cb>] elv_init_fq_data+0x32/0xe0
[<c0504317>] elevator_alloc+0x150/0x170
[<c0505393>] elevator_init+0x9d/0x100
[<c0507088>] blk_init_queue_node+0xc4/0xf7
[<c05070cb>] blk_init_queue+0x10/0x12
[<f81060fd>] __scsi_alloc_queue+0x1c/0xba [scsi_mod]
[<f81061b0>] scsi_alloc_queue+0x15/0x4e [scsi_mod]
[<f810803d>] scsi_alloc_sdev+0x154/0x1f5 [scsi_mod]
[<f8108387>] scsi_probe_and_add_lun+0x123/0xb5b [scsi_mod]
[<f8109847>] __scsi_add_device+0x8a/0xb0 [scsi_mod]
[<f816ad14>] ata_scsi_scan_host+0x77/0x141 [libata]
[<f816903f>] async_port_probe+0xa0/0xa9 [libata]
[<c044341f>] async_thread+0xe9/0x1c9
[<c043e204>] kthread+0x4a/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
}
... key at: [<c0c5ebd8>] __key.29462+0x0/0x8the second lock's dependencies:
-> (&q->__queue_lock){..-...} ops: 162810 {
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<f810672c>] scsi_device_unbusy+0x78/0x92 [scsi_mod]
[<f8101483>] scsi_finish_command+0x22/0xd4 [scsi_mod]
[<f8106fdb>] scsi_softirq_done+0xf9/0x101 [scsi_mod]
[<c050a936>] blk_done_softirq+0x5e/0x70
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<f8101337>] scsi_adjust_queue_depth+0x2a/0xc9 [scsi_mod]
[<f8108079>] scsi_alloc_sdev+0x190/0x1f5 [scsi_mod]
[<f8108387>] scsi_probe_and_add_lun+0x123/0xb5b [scsi_mod]
[<f8109847>] __scsi_add_device+0x8a/0xb0 [scsi_mod]
[<f816ad14>] ata_scsi_scan_host+0x77/0x141 [libata]
[<f816903f>] async_port_probe+0xa0/0xa9 [libata]
[<c044341f>] async_thread+0xe9/0x1c9
[<c043e204>] kthread+0x4a/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
}
... key at: [<c0c5e698>] __key.29749+0x0/0x8
-> (&ioc->lock){..-...} ops: 1032 {
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c050f0f0>] cic_free_func+0x26/0x64
[<c050ea90>] __call_for_each_cic+0x23/0x2e
[<c050eaad>] cfq_free_io_context+0x12/0x14
[<c050978c>] put_io_context+0x4b/0x66
[<c050f2a2>] cfq_put_request+0x42/0x5b
[<c0504629>] elv_put_request+0x30/0x33
[<c050678d>] __blk_put_request+0x8b/0xb8
[<c0506953>] end_that_request_last+0x199/0x1a1
[<c0506a0d>] blk_end_io+0x51/0x6f
[<c0506a64>] blk_end_request+0x11/0x13
[<f8106c9c>] scsi_io_completion+0x1d9/0x41f [scsi_mod]
[<f810152d>] scsi_finish_command+0xcc/0xd4 [scsi_mod]
[<f8106fdb>] scsi_softirq_done+0xf9/0x101 [scsi_mod]
[<c050a936>] blk_done_softirq+0x5e/0x70
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c050f9bf>] cfq_set_request+0x123/0x33d
[<c05052e6>] elv_set_request+0x43/0x53
[<c0506d44>] get_request+0x22e/0x33f
[<c0507498>] get_request_wait+0x137/0x15d
[<c0507501>] blk_get_request+0x43/0x73
[<f8106854>] scsi_execute+0x24/0x11c [scsi_mod]
[<f81069ff>] scsi_execute_req+0xb3/0x104 [scsi_mod]
[<f81084f8>] scsi_probe_and_add_lun+0x294/0xb5b [scsi_mod]
[<f8109847>] __scsi_add_device+0x8a/0xb0 [scsi_mod]
[<f816ad14>] ata_scsi_scan_host+0x77/0x141 [libata]
[<f816903f>] async_port_probe+0xa0/0xa9 [libata]
[<c044341f>] async_thread+0xe9/0x1c9
[<c043e204>] kthread+0x4a/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
}
... key at: [<c0c5e6ec>] __key.27747+0x0/0x8
-> (&rdp->lock){-.-...} ops: 168014 {
IN-HARDIRQ-W at:
[<c044d9e4>] __lock_acquire+0x49b/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c0461b2a>] rcu_check_callbacks+0x6a/0xa3
[<c043549a>] update_process_times+0x3d/0x53
[<c0447fe0>] tick_periodic+0x6b/0x77
[<c0448009>] tick_handle_periodic+0x1d/0x60
[<c063406e>] smp_apic_timer_interrupt+0x6e/0x81
[<c04033c7>] apic_timer_interrupt+0x2f/0x34
[<c042fbd7>] do_exit+0x53e/0x5b3
[<c043a9d8>] __request_module+0x0/0x100
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c04619db>] rcu_process_callbacks+0x2b/0x86
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c062c8ca>] rcu_online_cpu+0x3d/0x51
[<c062c910>] rcu_cpu_notify+0x32/0x43
[<c07b097f>] __rcu_init+0xf0/0x120
[<c07af027>] rcu_init+0x8/0x14
[<c079d6e1>] start_kernel+0x187/0x2fc
[<c079d06a>] __init_begin+0x6a/0x6f
[<ffffffff>] 0xffffffff
}
... key at: [<c0c2e52c>] __key.17543+0x0/0x8
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c046143d>] call_rcu+0x36/0x5b
[<c0517b45>] radix_tree_delete+0xe7/0x176
[<c050f0fe>] cic_free_func+0x34/0x64
[<c050ea90>] __call_for_each_cic+0x23/0x2e
[<c050eaad>] cfq_free_io_context+0x12/0x14
[<c050978c>] put_io_context+0x4b/0x66
[<c050984c>] exit_io_context+0x77/0x7b
[<c042fc24>] do_exit+0x58b/0x5b3
[<c04034ed>] kernel_thread_helper+0xd/0x10
[<ffffffff>] 0xffffffff... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae [<c044dbfd>] __lock_acquire+0x6b4/0x73e [<c044dd36>] lock_acquire+0xaf/0xcc [<c063056b>] _spin_lock_irqsave+0x33/0x43 [<c050f4a3>] cfq_cic_lookup+0xd9/0xef [<c050f674>] cfq_get_queue+0x92/0x2ba [<c050fb01>] cfq_set_request+0x265/0x33d [<c05052e6>] elv_set_request+0x43/0x53 [<c0506d44>] get_request+0x22e/0x33f [<c0507498>] get_request_wait+0x137/0x15d [<c0507501>] blk_get_request+0x43/0x73 [<f8106854>] scsi_execute+0x24/0x11c [scsi_mod] [<f81069ff>] scsi_execute_req+0xb3/0x104 [scsi_mod] [<f81084f8>] scsi_probe_and_add_lun+0x294/0xb5b [scsi_mod] [<f8109847>] __scsi_add_device+0x8a/0xb0 [scsi_mod] [<f816ad14>] ata_scsi_scan_host+0x77/0x141 [libata] [<f816903f>] async_port_probe+0xa0/0xa9 [libata] [<c044341f>] async_thread+0xe9/0x1c9 [<c043e204>] kthread+0x4a/0x72 [<c04034e7>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff
-> (&base->lock){..-...} ops: 348073 {
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c06304ea>] _spin_lock_irq+0x30/0x3f
[<c0434b8b>] run_timer_softirq+0x3c/0x1d1
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c0434e84>] lock_timer_base+0x24/0x43
[<c0434f3d>] mod_timer+0x46/0xcc
[<c07bd97a>] con_init+0xa4/0x20e
[<c07bd3b2>] console_init+0x12/0x20
[<c079d735>] start_kernel+0x1db/0x2fc
[<c079d06a>] __init_begin+0x6a/0x6f
[<ffffffff>] 0xffffffff
}
... key at: [<c082304c>] __key.23401+0x0/0x8
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c0434e84>] lock_timer_base+0x24/0x43
[<c0434f3d>] mod_timer+0x46/0xcc
[<c05075cb>] blk_plug_device+0x9a/0xdf
[<c05049e1>] __elv_add_request+0x86/0x96
[<c0509d52>] blk_execute_rq_nowait+0x5d/0x86
[<c0509e2e>] blk_execute_rq+0xb3/0xd5
[<f81068f5>] scsi_execute+0xc5/0x11c [scsi_mod]
[<f81069ff>] scsi_execute_req+0xb3/0x104 [scsi_mod]
[<f81084f8>] scsi_probe_and_add_lun+0x294/0xb5b [scsi_mod]
[<f8109847>] __scsi_add_device+0x8a/0xb0 [scsi_mod]
[<f816ad14>] ata_scsi_scan_host+0x77/0x141 [libata]
[<f816903f>] async_port_probe+0xa0/0xa9 [libata]
[<c044341f>] async_thread+0xe9/0x1c9
[<c043e204>] kthread+0x4a/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff -> (&sdev->list_lock){..-...} ops: 27612 {
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<f8101cb4>] scsi_put_command+0x17/0x57 [scsi_mod]
[<f810620f>] scsi_next_command+0x26/0x39 [scsi_mod]
[<f8106d02>] scsi_io_completion+0x23f/0x41f [scsi_mod]
[<f810152d>] scsi_finish_command+0xcc/0xd4 [scsi_mod]
[<f8106fdb>] scsi_softirq_done+0xf9/0x101 [scsi_mod]
[<c050a936>] blk_done_softirq+0x5e/0x70
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<f8101c64>] scsi_get_command+0x5c/0x95 [scsi_mod]
[<f81062b6>] scsi_get_cmd_from_req+0x26/0x50 [scsi_mod]
[<f8106594>] scsi_setup_blk_pc_cmnd+0x2b/0xd7 [scsi_mod]
[<f8106664>] scsi_prep_fn+0x24/0x33 [scsi_mod]
[<c0504712>] elv_next_request+0xe6/0x18d
[<f810704c>] scsi_request_fn+0x69/0x431 [scsi_mod]
[<c05072af>] __generic_unplug_device+0x2e/0x31
[<c0509d59>] blk_execute_rq_nowait+0x64/0x86
[<c0509e2e>] blk_execute_rq+0xb3/0xd5
[<f81068f5>] scsi_execute+0xc5/0x11c [scsi_mod]
[<f81069ff>] scsi_execute_req+0xb3/0x104 [scsi_mod]
[<f81084f8>] scsi_probe_and_add_lun+0x294/0xb5b [scsi_mod]
[<f8109847>] __scsi_add_device+0x8a/0xb0 [scsi_mod]
[<f816ad14>] ata_scsi_scan_host+0x77/0x141 [libata]
[<f816903f>] async_port_probe+0xa0/0xa9 [libata]
[<c044341f>] async_thread+0xe9/0x1c9
[<c043e204>] kthread+0x4a/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
}
... key at: [<f811916c>] __key.29786+0x0/0xffff2ebf [scsi_mod]
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<f8101c64>] scsi_get_command+0x5c/0x95 [scsi_mod]
[<f81062b6>] scsi_get_cmd_from_req+0x26/0x50 [scsi_mod]
[<f8106594>] scsi_setup_blk_pc_cmnd+0x2b/0xd7 [scsi_mod]
[<f8106664>] scsi_prep_fn+0x24/0x33 [scsi_mod]
[<c0504712>] elv_next_request+0xe6/0x18d
[<f810704c>] scsi_request_fn+0x69/0x431 [scsi_mod]
[<c05072af>] __generic_unplug_device+0x2e/0x31
[<c0509d59>] blk_execute_rq_nowait+0x64/0x86
[<c0509e2e>] blk_execute_rq+0xb3/0xd5
[<f81068f5>] scsi_execute+0xc5/0x11c [scsi_mod]
[<f81069ff>] scsi_execute_req+0xb3/0x104 [scsi_mod]
[<f81084f8>] scsi_probe_and_add_lun+0x294/0xb5b [scsi_mod]
[<f8109847>] __scsi_add_device+0x8a/0xb0 [scsi_mod]
[<f816ad14>] ata_scsi_scan_host+0x77/0x141 [libata]
[<f816903f>] async_port_probe+0xa0/0xa9 [libata]
[<c044341f>] async_thread+0xe9/0x1c9
[<c043e204>] kthread+0x4a/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff -> (&q->lock){-.-.-.} ops: 2105038 {
IN-HARDIRQ-W at:
[<c044d9e4>] __lock_acquire+0x49b/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c041ec0d>] complete+0x17/0x43
[<c062609b>] i8042_aux_test_irq+0x4c/0x65
[<c045e922>] handle_IRQ_event+0xa4/0x169
[<c04602ea>] handle_edge_irq+0xc9/0x10a
[<ffffffff>] 0xffffffff
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c041ec0d>] complete+0x17/0x43
[<c043c336>] wakeme_after_rcu+0x10/0x12
[<c0461a12>] rcu_process_callbacks+0x62/0x86
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
IN-RECLAIM_FS-W at:
[<c044dabd>] __lock_acquire+0x574/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c043e47b>] prepare_to_wait+0x1c/0x4a
[<c0485d3e>] kswapd+0xa7/0x51b
[<c043e204>] kthread+0x4a/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c06304ea>] _spin_lock_irq+0x30/0x3f
[<c062d811>] wait_for_common+0x2f/0xeb
[<c062d968>] wait_for_completion+0x17/0x19
[<c043e161>] kthread_create+0x6e/0xc7
[<c062b7eb>] migration_call+0x39/0x444
[<c07ae112>] migration_init+0x1d/0x4b
[<c040115c>] do_one_initcall+0x6a/0x16e
[<c079d44d>] kernel_init+0x4d/0x15a
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
}
... key at: [<c0823490>] __key.17681+0x0/0x8
-> (&rq->lock){-.-.-.} ops: 854341 {
IN-HARDIRQ-W at:
[<c044d9e4>] __lock_acquire+0x49b/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c0429f89>] scheduler_tick+0x39/0x19b
[<c04354a4>] update_process_times+0x47/0x53
[<c0447fe0>] tick_periodic+0x6b/0x77
[<c0448009>] tick_handle_periodic+0x1d/0x60
[<c0404ace>] timer_interrupt+0x3e/0x45
[<c045e922>] handle_IRQ_event+0xa4/0x169
[<c04603a3>] handle_level_irq+0x78/0xc1
[<ffffffff>] 0xffffffff
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c041ede7>] task_rq_lock+0x3b/0x62
[<c0426e41>] try_to_wake_up+0x75/0x2d4
[<c04270d7>] wake_up_process+0x14/0x16
[<c043507c>] process_timeout+0xd/0xf
[<c0434caa>] run_timer_softirq+0x15b/0x1d1
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
IN-RECLAIM_FS-W at:
[<c044dabd>] __lock_acquire+0x574/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c041ede7>] task_rq_lock+0x3b/0x62
[<c0427515>] set_cpus_allowed_ptr+0x1a/0xdd
[<c0485cf8>] kswapd+0x61/0x51b
[<c043e204>] kthread+0x4a/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c042398e>] rq_attach_root+0x17/0xa7
[<c07ae52c>] sched_init+0x240/0x33e
[<c079d661>] start_kernel+0x107/0x2fc
[<c079d06a>] __init_begin+0x6a/0x6f
[<ffffffff>] 0xffffffff
}
... key at: [<c0800518>] __key.46938+0x0/0x8
-> (&vec->lock){-.-...} ops: 34058 {
IN-HARDIRQ-W at:
[<c044d9e4>] __lock_acquire+0x49b/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c047ad3b>] cpupri_set+0x51/0xba
[<c04219ee>] __enqueue_rt_entity+0xe2/0x1c8
[<c0421e18>] enqueue_rt_entity+0x19/0x23
[<c0428a52>] enqueue_task_rt+0x24/0x51
[<c041e03b>] enqueue_task+0x64/0x70
[<c041e06b>] activate_task+0x24/0x2a
[<c0426f9e>] try_to_wake_up+0x1d2/0x2d4
[<c04270d7>] wake_up_process+0x14/0x16
[<c04408b6>] hrtimer_wakeup+0x1d/0x21
[<c0440922>] __run_hrtimer+0x68/0x98
[<c04411ca>] hrtimer_interrupt+0x101/0x153
[<c063406e>] smp_apic_timer_interrupt+0x6e/0x81
[<c04033c7>] apic_timer_interrupt+0x2f/0x34
[<c0401c4f>] cpu_idle+0x53/0x85
[<c061fc80>] rest_init+0x6c/0x6e
[<c079d851>] start_kernel+0x2f7/0x2fc
[<c079d06a>] __init_begin+0x6a/0x6f
[<ffffffff>] 0xffffffff
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c047ad3b>] cpupri_set+0x51/0xba
[<c04219ee>] __enqueue_rt_entity+0xe2/0x1c8
[<c0421e18>] enqueue_rt_entity+0x19/0x23
[<c0428a52>] enqueue_task_rt+0x24/0x51
[<c041e03b>] enqueue_task+0x64/0x70
[<c041e06b>] activate_task+0x24/0x2a
[<c0426f9e>] try_to_wake_up+0x1d2/0x2d4
[<c04270d7>] wake_up_process+0x14/0x16
[<c042737c>] rebalance_domains+0x2a3/0x3ac
[<c0429a06>] run_rebalance_domains+0x32/0xaa
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c047ad74>] cpupri_set+0x8a/0xba
[<c04216f2>] rq_online_rt+0x5e/0x61
[<c041dd3a>] set_rq_online+0x40/0x4a
[<c04239fb>] rq_attach_root+0x84/0xa7
[<c07ae52c>] sched_init+0x240/0x33e
[<c079d661>] start_kernel+0x107/0x2fc
[<c079d06a>] __init_begin+0x6a/0x6f
[<ffffffff>] 0xffffffff
}
... key at: [<c0c525d0>] __key.14261+0x0/0x10
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c047ad74>] cpupri_set+0x8a/0xba
[<c04216f2>] rq_online_rt+0x5e/0x61
[<c041dd3a>] set_rq_online+0x40/0x4a
[<c04239fb>] rq_attach_root+0x84/0xa7
[<c07ae52c>] sched_init+0x240/0x33e
[<c079d661>] start_kernel+0x107/0x2fc
[<c079d06a>] __init_begin+0x6a/0x6f
[<ffffffff>] 0xffffffff -> (&rt_b->rt_runtime_lock){-.-...} ops: 336 {
IN-HARDIRQ-W at:
[<c044d9e4>] __lock_acquire+0x49b/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c0421a75>] __enqueue_rt_entity+0x169/0x1c8
[<c0421e18>] enqueue_rt_entity+0x19/0x23
[<c0428a52>] enqueue_task_rt+0x24/0x51
[<c041e03b>] enqueue_task+0x64/0x70
[<c041e06b>] activate_task+0x24/0x2a
[<c0426f9e>] try_to_wake_up+0x1d2/0x2d4
[<c04270d7>] wake_up_process+0x14/0x16
[<c04408b6>] hrtimer_wakeup+0x1d/0x21
[<c0440922>] __run_hrtimer+0x68/0x98
[<c04411ca>] hrtimer_interrupt+0x101/0x153
[<c063406e>] smp_apic_timer_interrupt+0x6e/0x81
[<c04033c7>] apic_timer_interrupt+0x2f/0x34
[<c0401c4f>] cpu_idle+0x53/0x85
[<c061fc80>] rest_init+0x6c/0x6e
[<c079d851>] start_kernel+0x2f7/0x2fc
[<c079d06a>] __init_begin+0x6a/0x6f
[<ffffffff>] 0xffffffff
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c0421a75>] __enqueue_rt_entity+0x169/0x1c8
[<c0421e18>] enqueue_rt_entity+0x19/0x23
[<c0428a52>] enqueue_task_rt+0x24/0x51
[<c041e03b>] enqueue_task+0x64/0x70
[<c041e06b>] activate_task+0x24/0x2a
[<c0426f9e>] try_to_wake_up+0x1d2/0x2d4
[<c04270d7>] wake_up_process+0x14/0x16
[<c042737c>] rebalance_domains+0x2a3/0x3ac
[<c0429a06>] run_rebalance_domains+0x32/0xaa
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c0421a75>] __enqueue_rt_entity+0x169/0x1c8
[<c0421e18>] enqueue_rt_entity+0x19/0x23
[<c0428a52>] enqueue_task_rt+0x24/0x51
[<c041e03b>] enqueue_task+0x64/0x70
[<c041e06b>] activate_task+0x24/0x2a
[<c0426f9e>] try_to_wake_up+0x1d2/0x2d4
[<c04270d7>] wake_up_process+0x14/0x16
[<c062b86b>] migration_call+0xb9/0x444
[<c07ae130>] migration_init+0x3b/0x4b
[<c040115c>] do_one_initcall+0x6a/0x16e
[<c079d44d>] kernel_init+0x4d/0x15a
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
}
... key at: [<c0800504>] __key.37924+0x0/0x8
-> (&cpu_base->lock){-.-...} ops: 950512 {
IN-HARDIRQ-W at:
[<c044d9e4>] __lock_acquire+0x49b/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c0440a3a>] hrtimer_run_queues+0xe8/0x131
[<c0435151>] run_local_timers+0xd/0x1e
[<c0435486>] update_process_times+0x29/0x53
[<c0447fe0>] tick_periodic+0x6b/0x77
[<c0448009>] tick_handle_periodic+0x1d/0x60
[<c063406e>] smp_apic_timer_interrupt+0x6e/0x81
[<c04033c7>] apic_timer_interrupt+0x2f/0x34
[<c04082c7>] arch_dup_task_struct+0x19/0x81
[<c042ac1c>] copy_process+0xab/0x115f
[<c042be78>] do_fork+0x129/0x2c5
[<c0401698>] kernel_thread+0x7f/0x87
[<c043e0b3>] kthreadd+0xa3/0xe3
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c0440b98>] lock_hrtimer_base+0x1d/0x38
[<c0440ca9>] __hrtimer_start_range_ns+0x1f/0x232
[<c0440ee7>] hrtimer_start_range_ns+0x15/0x17
[<c0448ef1>] tick_setup_sched_timer+0xf6/0x124
[<c0441558>] hrtimer_run_pending+0xb0/0xe8
[<c0434b76>] run_timer_softirq+0x27/0x1d1
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c0440b98>] lock_hrtimer_base+0x1d/0x38
[<c0440ca9>] __hrtimer_start_range_ns+0x1f/0x232
[<c0421ab1>] __enqueue_rt_entity+0x1a5/0x1c8
[<c0421e18>] enqueue_rt_entity+0x19/0x23
[<c0428a52>] enqueue_task_rt+0x24/0x51
[<c041e03b>] enqueue_task+0x64/0x70
[<c041e06b>] activate_task+0x24/0x2a
[<c0426f9e>] try_to_wake_up+0x1d2/0x2d4
[<c04270d7>] wake_up_process+0x14/0x16
[<c062b86b>] migration_call+0xb9/0x444
[<c07ae130>] migration_init+0x3b/0x4b
[<c040115c>] do_one_initcall+0x6a/0x16e
[<c079d44d>] kernel_init+0x4d/0x15a
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
}
... key at: [<c08234b8>] __key.20063+0x0/0x8
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c0440b98>] lock_hrtimer_base+0x1d/0x38
[<c0440ca9>] __hrtimer_start_range_ns+0x1f/0x232
[<c0421ab1>] __enqueue_rt_entity+0x1a5/0x1c8
[<c0421e18>] enqueue_rt_entity+0x19/0x23
[<c0428a52>] enqueue_task_rt+0x24/0x51
[<c041e03b>] enqueue_task+0x64/0x70
[<c041e06b>] activate_task+0x24/0x2a
[<c0426f9e>] try_to_wake_up+0x1d2/0x2d4
[<c04270d7>] wake_up_process+0x14/0x16
[<c062b86b>] migration_call+0xb9/0x444
[<c07ae130>] migration_init+0x3b/0x4b
[<c040115c>] do_one_initcall+0x6a/0x16e
[<c079d44d>] kernel_init+0x4d/0x15a
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff -> (&rt_rq->rt_runtime_lock){-.....} ops: 17587 {
IN-HARDIRQ-W at:
[<c044d9e4>] __lock_acquire+0x49b/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c0421efc>] sched_rt_period_timer+0xda/0x24e
[<c0440922>] __run_hrtimer+0x68/0x98
[<c04411ca>] hrtimer_interrupt+0x101/0x153
[<c063406e>] smp_apic_timer_interrupt+0x6e/0x81
[<c04033c7>] apic_timer_interrupt+0x2f/0x34
[<c0452203>] each_symbol_in_section+0x27/0x57
[<c045225a>] each_symbol+0x27/0x113
[<c0452373>] find_symbol+0x2d/0x51
[<c0454a7a>] load_module+0xaec/0x10eb
[<c04550bf>] sys_init_module+0x46/0x19b
[<c0402a68>] sysenter_do_call+0x12/0x36
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c0421c41>] update_curr_rt+0x13a/0x20d
[<c0421dd8>] dequeue_task_rt+0x13/0x3a
[<c041df9e>] dequeue_task+0xff/0x10e
[<c041dfd1>] deactivate_task+0x24/0x2a
[<c062db54>] __schedule+0x162/0x991
[<c062e39a>] schedule+0x17/0x30
[<c0426c54>] migration_thread+0x175/0x203
[<c043e204>] kthread+0x4a/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
}
... key at: [<c080050c>] __key.46863+0x0/0x8
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c041ee73>] __enable_runtime+0x43/0xb3
[<c04216d8>] rq_online_rt+0x44/0x61
[<c041dd3a>] set_rq_online+0x40/0x4a
[<c062b8a5>] migration_call+0xf3/0x444
[<c063291c>] notifier_call_chain+0x2b/0x4a
[<c0441e22>] __raw_notifier_call_chain+0x13/0x15
[<c0441e35>] raw_notifier_call_chain+0x11/0x13
[<c062bd2f>] _cpu_up+0xc3/0xf6
[<c062bdac>] cpu_up+0x4a/0x5a
[<c079d49a>] kernel_init+0x9a/0x15a
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae [<c044dbfd>] __lock_acquire+0x6b4/0x73e [<c044dd36>] lock_acquire+0xaf/0xcc [<c0630340>] _spin_lock+0x2a/0x39 [<c0421a75>] __enqueue_rt_entity+0x169/0x1c8 [<c0421e18>] enqueue_rt_entity+0x19/0x23 [<c0428a52>] enqueue_task_rt+0x24/0x51 [<c041e03b>] enqueue_task+0x64/0x70 [<c041e06b>] activate_task+0x24/0x2a [<c0426f9e>] try_to_wake_up+0x1d2/0x2d4 [<c04270d7>] wake_up_process+0x14/0x16 [<c062b86b>] migration_call+0xb9/0x444 [<c07ae130>] migration_init+0x3b/0x4b [<c040115c>] do_one_initcall+0x6a/0x16e [<c079d44d>] kernel_init+0x4d/0x15a [<c04034e7>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae [<c044dbfd>] __lock_acquire+0x6b4/0x73e [<c044dd36>] lock_acquire+0xaf/0xcc [<c0630340>] _spin_lock+0x2a/0x39 [<c0421c41>] update_curr_rt+0x13a/0x20d [<c0421dd8>] dequeue_task_rt+0x13/0x3a [<c041df9e>] dequeue_task+0xff/0x10e [<c041dfd1>] deactivate_task+0x24/0x2a [<c062db54>] __schedule+0x162/0x991 [<c062e39a>] schedule+0x17/0x30 [<c0426c54>] migration_thread+0x175/0x203 [<c043e204>] kthread+0x4a/0x72 [<c04034e7>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff
-> (&sig->cputimer.lock){......} ops: 1949 {
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c043f03e>] thread_group_cputimer+0x29/0x90
[<c044004c>] posix_cpu_timers_exit_group+0x16/0x39
[<c042e5f0>] release_task+0xa2/0x376
[<c042fbe1>] do_exit+0x548/0x5b3
[<c043a9d8>] __request_module+0x0/0x100
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
}
... key at: [<c08014ac>] __key.15480+0x0/0x8
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c041f43a>] update_curr+0xef/0x107
[<c042131b>] enqueue_entity+0x1a/0x1c6
[<c0421535>] enqueue_task_fair+0x24/0x3e
[<c041e03b>] enqueue_task+0x64/0x70
[<c041e06b>] activate_task+0x24/0x2a
[<c0426f9e>] try_to_wake_up+0x1d2/0x2d4
[<c04270b0>] default_wake_function+0x10/0x12
[<c041d785>] __wake_up_common+0x34/0x5f
[<c041ec26>] complete+0x30/0x43
[<c043e1e8>] kthread+0x2e/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
-> (&rq->lock/1){..-...} ops: 3217 {
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630305>] _spin_lock_nested+0x2d/0x3e
[<c0422cb4>] double_rq_lock+0x4b/0x7d
[<c0427274>] rebalance_domains+0x19b/0x3ac
[<c0429a06>] run_rebalance_domains+0x32/0xaa
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630305>] _spin_lock_nested+0x2d/0x3e
[<c0422cb4>] double_rq_lock+0x4b/0x7d
[<c0427274>] rebalance_domains+0x19b/0x3ac
[<c0429a06>] run_rebalance_domains+0x32/0xaa
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
}
... key at: [<c0800519>] __key.46938+0x1/0x8
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c0421c41>] update_curr_rt+0x13a/0x20d
[<c0421dd8>] dequeue_task_rt+0x13/0x3a
[<c041df9e>] dequeue_task+0xff/0x10e
[<c041dfd1>] deactivate_task+0x24/0x2a
[<c0427b1b>] push_rt_task+0x189/0x1f7
[<c0427b9b>] push_rt_tasks+0x12/0x19
[<c0427bb9>] post_schedule_rt+0x17/0x21
[<c0425a68>] finish_task_switch+0x83/0xc0
[<c062e339>] __schedule+0x947/0x991
[<c062e39a>] schedule+0x17/0x30
[<c0426c54>] migration_thread+0x175/0x203
[<c043e204>] kthread+0x4a/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae [<c044dbfd>] __lock_acquire+0x6b4/0x73e [<c044dd36>] lock_acquire+0xaf/0xcc [<c063056b>] _spin_lock_irqsave+0x33/0x43 [<c047ad3b>] cpupri_set+0x51/0xba [<c04219ee>] __enqueue_rt_entity+0xe2/0x1c8 [<c0421e18>] enqueue_rt_entity+0x19/0x23 [<c0428a52>] enqueue_task_rt+0x24/0x51 [<c041e03b>] enqueue_task+0x64/0x70 [<c041e06b>] activate_task+0x24/0x2a [<c0427b33>] push_rt_task+0x1a1/0x1f7 [<c0427b9b>] push_rt_tasks+0x12/0x19 [<c0427bb9>] post_schedule_rt+0x17/0x21 [<c0425a68>] finish_task_switch+0x83/0xc0 [<c062e339>] __schedule+0x947/0x991 [<c062e39a>] schedule+0x17/0x30 [<c0426c54>] migration_thread+0x175/0x203 [<c043e204>] kthread+0x4a/0x72 [<c04034e7>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae [<c044dbfd>] __lock_acquire+0x6b4/0x73e [<c044dd36>] lock_acquire+0xaf/0xcc [<c0630305>] _spin_lock_nested+0x2d/0x3e [<c0422cb4>] double_rq_lock+0x4b/0x7d [<c0427274>] rebalance_domains+0x19b/0x3ac [<c0429a06>] run_rebalance_domains+0x32/0xaa [<c0431379>] __do_softirq+0xb8/0x180 [<ffffffff>] 0xffffffff
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae [<c044dbfd>] __lock_acquire+0x6b4/0x73e [<c044dd36>] lock_acquire+0xaf/0xcc [<c0630340>] _spin_lock+0x2a/0x39 [<c041ede7>] task_rq_lock+0x3b/0x62 [<c0426e41>] try_to_wake_up+0x75/0x2d4 [<c04270b0>] default_wake_function+0x10/0x12 [<c041d785>] __wake_up_common+0x34/0x5f [<c041ec26>] complete+0x30/0x43 [<c043e0cc>] kthreadd+0xbc/0xe3 [<c04034e7>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff
-> (&ep->lock){......} ops: 110 {
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c04ca381>] sys_epoll_ctl+0x232/0x3f6
[<c0402a68>] sysenter_do_call+0x12/0x36
[<ffffffff>] 0xffffffff
}
... key at: [<c0c5be90>] __key.22301+0x0/0x10
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c041ede7>] task_rq_lock+0x3b/0x62
[<c0426e41>] try_to_wake_up+0x75/0x2d4
[<c04270b0>] default_wake_function+0x10/0x12
[<c041d785>] __wake_up_common+0x34/0x5f
[<c041d7c6>] __wake_up_locked+0x16/0x1a
[<c04ca7f5>] ep_poll_callback+0x7c/0xb6
[<c041d785>] __wake_up_common+0x34/0x5f
[<c041ec70>] __wake_up_sync_key+0x37/0x4a
[<c05cbefa>] sock_def_readable+0x42/0x71
[<c061c8b1>] unix_stream_connect+0x2f3/0x368
[<c05c830a>] sys_connect+0x59/0x76
[<c05c963f>] sys_socketcall+0x76/0x172
[<c0402a68>] sysenter_do_call+0x12/0x36
[<ffffffff>] 0xffffffff... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae [<c044dbfd>] __lock_acquire+0x6b4/0x73e [<c044dd36>] lock_acquire+0xaf/0xcc [<c063056b>] _spin_lock_irqsave+0x33/0x43 [<c04ca797>] ep_poll_callback+0x1e/0xb6 [<c041d785>] __wake_up_common+0x34/0x5f [<c041ec70>] __wake_up_sync_key+0x37/0x4a [<c05cbefa>] sock_def_readable+0x42/0x71 [<c061c8b1>] unix_stream_connect+0x2f3/0x368 [<c05c830a>] sys_connect+0x59/0x76 [<c05c963f>] sys_socketcall+0x76/0x172 [<c0402a68>] sysenter_do_call+0x12/0x36 [<ffffffff>] 0xffffffff
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae [<c044dbfd>] __lock_acquire+0x6b4/0x73e [<c044dd36>] lock_acquire+0xaf/0xcc [<c063056b>] _spin_lock_irqsave+0x33/0x43 [<c041ec0d>] complete+0x17/0x43 [<c0509cf2>] blk_end_sync_rq+0x2a/0x2d [<c0506935>] end_that_request_last+0x17b/0x1a1 [<c0506a0d>] blk_end_io+0x51/0x6f [<c0506a64>] blk_end_request+0x11/0x13 [<f8106c9c>] scsi_io_completion+0x1d9/0x41f [scsi_mod] [<f810152d>] scsi_finish_command+0xcc/0xd4 [scsi_mod] [<f8106fdb>] scsi_softirq_done+0xf9/0x101 [scsi_mod] [<c050a936>] blk_done_softirq+0x5e/0x70 [<c0431379>] __do_softirq+0xb8/0x180 [<ffffffff>] 0xffffffff
-> (&n->list_lock){..-...} ops: 49241 {
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c049bd18>] add_partial+0x16/0x40
[<c049d0d4>] __slab_free+0x96/0x28f
[<c049df5c>] kmem_cache_free+0x8c/0xf2
[<c04a5ce9>] file_free_rcu+0x35/0x38
[<c0461a12>] rcu_process_callbacks+0x62/0x86
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c049bd18>] add_partial+0x16/0x40
[<c049d0d4>] __slab_free+0x96/0x28f
[<c049df5c>] kmem_cache_free+0x8c/0xf2
[<c0514eda>] ida_get_new_above+0x13b/0x155
[<c0514f00>] ida_get_new+0xc/0xe
[<c04a628b>] set_anon_super+0x39/0xa3
[<c04a68c6>] sget+0x2f3/0x386
[<c04a7365>] get_sb_single+0x24/0x8f
[<c04e034c>] sysfs_get_sb+0x18/0x1a
[<c04a6dd1>] vfs_kern_mount+0x40/0x7b
[<c04a6e21>] kern_mount_data+0x15/0x17
[<c07b5ff6>] sysfs_init+0x50/0x9c
[<c07b4ac9>] mnt_init+0x8c/0x1e4
[<c07b4737>] vfs_caches_init+0xd8/0xea
[<c079d815>] start_kernel+0x2bb/0x2fc
[<c079d06a>] __init_begin+0x6a/0x6f
[<ffffffff>] 0xffffffff
}
... key at: [<c0c5a424>] __key.25358+0x0/0x8
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c049cc45>] __slab_alloc+0xf6/0x4ef
[<c049d333>] kmem_cache_alloc+0x66/0x11f
[<f810189b>] scsi_pool_alloc_command+0x20/0x4c [scsi_mod]
[<f81018de>] scsi_host_alloc_command+0x17/0x4f [scsi_mod]
[<f810192b>] __scsi_get_command+0x15/0x71 [scsi_mod]
[<f8101c41>] scsi_get_command+0x39/0x95 [scsi_mod]
[<f81062b6>] scsi_get_cmd_from_req+0x26/0x50 [scsi_mod]
[<f8106594>] scsi_setup_blk_pc_cmnd+0x2b/0xd7 [scsi_mod]
[<f8106664>] scsi_prep_fn+0x24/0x33 [scsi_mod]
[<c0504712>] elv_next_request+0xe6/0x18d
[<f810704c>] scsi_request_fn+0x69/0x431 [scsi_mod]
[<c05072af>] __generic_unplug_device+0x2e/0x31
[<c0509d59>] blk_execute_rq_nowait+0x64/0x86
[<c0509e2e>] blk_execute_rq+0xb3/0xd5
[<f81068f5>] scsi_execute+0xc5/0x11c [scsi_mod]
[<f81069ff>] scsi_execute_req+0xb3/0x104 [scsi_mod]
[<f812b40d>] sd_revalidate_disk+0x1a3/0xf64 [sd_mod]
[<f812d52f>] sd_probe_async+0x146/0x22d [sd_mod]
[<c044341f>] async_thread+0xe9/0x1c9
[<c043e204>] kthread+0x4a/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff -> (&cwq->lock){-.-...} ops: 30335 {
IN-HARDIRQ-W at:
[<c044d9e4>] __lock_acquire+0x49b/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c043b54b>] __queue_work+0x14/0x30
[<c043b5ce>] queue_work_on+0x3a/0x46
[<c043b617>] queue_work+0x26/0x4a
[<c043b64f>] schedule_work+0x14/0x16
[<c057a367>] schedule_console_callback+0x12/0x14
[<c05788ed>] kbd_event+0x595/0x600
[<c05b3d15>] input_pass_event+0x56/0x7e
[<c05b4702>] input_handle_event+0x314/0x334
[<c05b4f1e>] input_event+0x50/0x63
[<c05b9bd4>] atkbd_interrupt+0x209/0x4e9
[<c05b1793>] serio_interrupt+0x38/0x6e
[<c05b24e8>] i8042_interrupt+0x1db/0x1ec
[<c045e922>] handle_IRQ_event+0xa4/0x169
[<c04602ea>] handle_edge_irq+0xc9/0x10a
[<ffffffff>] 0xffffffff
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c043b54b>] __queue_work+0x14/0x30
[<c043b590>] delayed_work_timer_fn+0x29/0x2d
[<c0434caa>] run_timer_softirq+0x15b/0x1d1
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c043b54b>] __queue_work+0x14/0x30
[<c043b5ce>] queue_work_on+0x3a/0x46
[<c043b617>] queue_work+0x26/0x4a
[<c043a7b3>] call_usermodehelper_exec+0x83/0xd0
[<c051631a>] kobject_uevent_env+0x351/0x385
[<c0516358>] kobject_uevent+0xa/0xc
[<c0515a0e>] kset_register+0x2e/0x34
[<c0590f18>] bus_register+0xed/0x23d
[<c07bea09>] platform_bus_init+0x23/0x38
[<c07beb77>] driver_init+0x1c/0x28
[<c079d4f6>] kernel_init+0xf6/0x15a
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
}
... key at: [<c08230a8>] __key.23814+0x0/0x8
-> (&workqueue_cpu_stat(cpu)->lock){-.-...} ops: 20397 {
IN-HARDIRQ-W at:
[<c044d9e4>] __lock_acquire+0x49b/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c0474909>] probe_workqueue_insertion+0x33/0x81
[<c043acf3>] insert_work+0x3f/0x9b
[<c043b559>] __queue_work+0x22/0x30
[<c043b5ce>] queue_work_on+0x3a/0x46
[<c043b617>] queue_work+0x26/0x4a
[<c043b64f>] schedule_work+0x14/0x16
[<c057a367>] schedule_console_callback+0x12/0x14
[<c05788ed>] kbd_event+0x595/0x600
[<c05b3d15>] input_pass_event+0x56/0x7e
[<c05b4702>] input_handle_event+0x314/0x334
[<c05b4f1e>] input_event+0x50/0x63
[<c05b9bd4>] atkbd_interrupt+0x209/0x4e9
[<c05b1793>] serio_interrupt+0x38/0x6e
[<c05b24e8>] i8042_interrupt+0x1db/0x1ec
[<c045e922>] handle_IRQ_event+0xa4/0x169
[<c04602ea>] handle_edge_irq+0xc9/0x10a
[<ffffffff>] 0xffffffff
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c0474909>] probe_workqueue_insertion+0x33/0x81
[<c043acf3>] insert_work+0x3f/0x9b
[<c043b559>] __queue_work+0x22/0x30
[<c043b590>] delayed_work_timer_fn+0x29/0x2d
[<c0434caa>] run_timer_softirq+0x15b/0x1d1
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c04747eb>] probe_workqueue_creation+0xc9/0x10a
[<c043abcb>] create_workqueue_thread+0x87/0xb0
[<c043b12f>] __create_workqueue_key+0x16d/0x1b2
[<c07aeedb>] init_workqueues+0x61/0x73
[<c079d4e7>] kernel_init+0xe7/0x15a
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
}
... key at: [<c0c52574>] __key.23424+0x0/0x8
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c0474909>] probe_workqueue_insertion+0x33/0x81
[<c043acf3>] insert_work+0x3f/0x9b
[<c043b559>] __queue_work+0x22/0x30
[<c043b5ce>] queue_work_on+0x3a/0x46
[<c043b617>] queue_work+0x26/0x4a
[<c043a7b3>] call_usermodehelper_exec+0x83/0xd0
[<c051631a>] kobject_uevent_env+0x351/0x385
[<c0516358>] kobject_uevent+0xa/0xc
[<c0515a0e>] kset_register+0x2e/0x34
[<c0590f18>] bus_register+0xed/0x23d
[<c07bea09>] platform_bus_init+0x23/0x38
[<c07beb77>] driver_init+0x1c/0x28
[<c079d4f6>] kernel_init+0xf6/0x15a
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae [<c044dbfd>] __lock_acquire+0x6b4/0x73e [<c044dd36>] lock_acquire+0xaf/0xcc [<c063056b>] _spin_lock_irqsave+0x33/0x43 [<c041ecaf>] __wake_up+0x1a/0x40 [<c043ad46>] insert_work+0x92/0x9b [<c043b559>] __queue_work+0x22/0x30 [<c043b5ce>] queue_work_on+0x3a/0x46 [<c043b617>] queue_work+0x26/0x4a [<c043a7b3>] call_usermodehelper_exec+0x83/0xd0 [<c051631a>] kobject_uevent_env+0x351/0x385 [<c0516358>] kobject_uevent+0xa/0xc [<c0515a0e>] kset_register+0x2e/0x34 [<c0590f18>] bus_register+0xed/0x23d [<c07bea09>] platform_bus_init+0x23/0x38 [<c07beb77>] driver_init+0x1c/0x28 [<c079d4f6>] kernel_init+0xf6/0x15a [<c04034e7>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae [<c044dbfd>] __lock_acquire+0x6b4/0x73e [<c044dd36>] lock_acquire+0xaf/0xcc [<c063056b>] _spin_lock_irqsave+0x33/0x43 [<c043b54b>] __queue_work+0x14/0x30 [<c043b5ce>] queue_work_on+0x3a/0x46 [<c043b617>] queue_work+0x26/0x4a [<c0505679>] kblockd_schedule_work+0x12/0x14 [<c05113bb>] elv_schedule_dispatch+0x41/0x48 [<c0513377>] elv_ioq_completed_request+0x2dc/0x2fe [<c05045aa>] elv_completed_request+0x48/0x97 [<c0506738>] __blk_put_request+0x36/0xb8 [<c0506953>] end_that_request_last+0x199/0x1a1 [<c0506a0d>] blk_end_io+0x51/0x6f [<c0506a64>] blk_end_request+0x11/0x13 [<f8106c9c>] scsi_io_completion+0x1d9/0x41f [scsi_mod] [<f810152d>] scsi_finish_command+0xcc/0xd4 [scsi_mod] [<f8106fdb>] scsi_softirq_done+0xf9/0x101 [scsi_mod] [<c050a936>] blk_done_softirq+0x5e/0x70 [<c0431379>] __do_softirq+0xb8/0x180 [<ffffffff>] 0xffffffff
-> (&zone->lock){..-...} ops: 80266 {
IN-SOFTIRQ-W at:
[<c044da08>] __lock_acquire+0x4bf/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c047fc71>] __free_pages_ok+0x167/0x321
[<c04800ce>] __free_pages+0x29/0x2b
[<c049c7c1>] __free_slab+0xb2/0xba
[<c049c800>] discard_slab+0x37/0x39
[<c049d15c>] __slab_free+0x11e/0x28f
[<c049df5c>] kmem_cache_free+0x8c/0xf2
[<c042ab6e>] free_task+0x31/0x34
[<c042c37b>] __put_task_struct+0xd3/0xd8
[<c042e072>] delayed_put_task_struct+0x60/0x64
[<c0461a12>] rcu_process_callbacks+0x62/0x86
[<c0431379>] __do_softirq+0xb8/0x180
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c0630340>] _spin_lock+0x2a/0x39
[<c047f7b6>] free_pages_bulk+0x21/0x1a1
[<c047ffcf>] free_hot_cold_page+0x181/0x20f
[<c04800a3>] free_hot_page+0xf/0x11
[<c04800c5>] __free_pages+0x20/0x2b
[<c07c4d96>] __free_pages_bootmem+0x6d/0x71
[<c07b2244>] free_all_bootmem_core+0xd2/0x177
[<c07b22f6>] free_all_bootmem+0xd/0xf
[<c07ad21a>] mem_init+0x28/0x28c
[<c079d7b1>] start_kernel+0x257/0x2fc
[<c079d06a>] __init_begin+0x6a/0x6f
[<ffffffff>] 0xffffffff
}
... key at: [<c0c52628>] __key.30749+0x0/0x8
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c048035e>] get_page_from_freelist+0x236/0x3e3
[<c04805f4>] __alloc_pages_internal+0xce/0x371
[<c049cce6>] __slab_alloc+0x197/0x4ef
[<c049d333>] kmem_cache_alloc+0x66/0x11f
[<c047d96b>] mempool_alloc_slab+0x13/0x15
[<c047da5c>] mempool_alloc+0x3a/0xd5
[<f81063cc>] scsi_sg_alloc+0x47/0x4a [scsi_mod]
[<c051cd02>] __sg_alloc_table+0x48/0xc7
[<f8106325>] scsi_init_sgtable+0x2c/0x8c [scsi_mod]
[<f81064e7>] scsi_init_io+0x19/0x9b [scsi_mod]
[<f8106abf>] scsi_setup_fs_cmnd+0x6f/0x73 [scsi_mod]
[<f812ca73>] sd_prep_fn+0x6a/0x7d4 [sd_mod]
[<c0504712>] elv_next_request+0xe6/0x18d
[<f810704c>] scsi_request_fn+0x69/0x431 [scsi_mod]
[<c05072af>] __generic_unplug_device+0x2e/0x31
[<c05072db>] blk_start_queueing+0x29/0x2b
[<c05137b8>] elv_ioq_request_add+0x2be/0x393
[<c05048cd>] elv_insert+0x114/0x1a2
[<c05049ec>] __elv_add_request+0x91/0x96
[<c0507a00>] __make_request+0x365/0x397
[<c050635a>] generic_make_request+0x342/0x3ce
[<c0507b21>] submit_bio+0xef/0xfa
[<c04c6c4e>] mpage_bio_submit+0x21/0x26
[<c04c7b7f>] mpage_readpages+0xa3/0xad
[<f80c1ea8>] ext3_readpages+0x19/0x1b [ext3]
[<c048275e>] __do_page_cache_readahead+0xfd/0x166
[<c0482b42>] do_page_cache_readahead+0x44/0x52
[<c047d665>] filemap_fault+0x197/0x3ae
[<c048b9ea>] __do_fault+0x40/0x37b
[<c048d43f>] handle_mm_fault+0x2bb/0x646
[<c063273c>] do_page_fault+0x29c/0x2fd
[<c0630b4a>] error_code+0x72/0x78
[<ffffffff>] 0xffffffff -> (&page_address_htable[i].lock){......} ops: 6802 {
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c048af69>] page_address+0x50/0xa6
[<c048b0e7>] kmap_high+0x21/0x175
[<c041b7ef>] kmap+0x4e/0x5b
[<c04abb36>] page_getlink+0x37/0x59
[<c04abb75>] page_follow_link_light+0x1d/0x2b
[<c04ad4d0>] __link_path_walk+0x3d1/0xa71
[<c04adbae>] path_walk+0x3e/0x77
[<c04add0e>] do_path_lookup+0xeb/0x105
[<c04ae6f2>] path_lookup_open+0x48/0x7a
[<c04a8e96>] open_exec+0x25/0xf4
[<c04a9c2d>] do_execve+0xfa/0x2cc
[<c04015c0>] sys_execve+0x2b/0x54
[<c0402ae9>] syscall_call+0x7/0xb
[<ffffffff>] 0xffffffff
}
... key at: [<c0c5288c>] __key.28547+0x0/0x14
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c048af69>] page_address+0x50/0xa6
[<c05078a1>] __make_request+0x206/0x397
[<c050635a>] generic_make_request+0x342/0x3ce
[<c0507b21>] submit_bio+0xef/0xfa
[<c04c6c4e>] mpage_bio_submit+0x21/0x26
[<c04c78b8>] do_mpage_readpage+0x471/0x5e5
[<c04c7b55>] mpage_readpages+0x79/0xad
[<f80c1ea8>] ext3_readpages+0x19/0x1b [ext3]
[<c048275e>] __do_page_cache_readahead+0xfd/0x166
[<c0482b42>] do_page_cache_readahead+0x44/0x52
[<c047d665>] filemap_fault+0x197/0x3ae
[<c048b9ea>] __do_fault+0x40/0x37b
[<c048d43f>] handle_mm_fault+0x2bb/0x646
[<c063273c>] do_page_fault+0x29c/0x2fd
[<c0630b4a>] error_code+0x72/0x78
[<ffffffff>] 0xffffffff... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae [<c044dbfd>] __lock_acquire+0x6b4/0x73e [<c044dd36>] lock_acquire+0xaf/0xcc [<c0630340>] _spin_lock+0x2a/0x39 [<c046143d>] call_rcu+0x36/0x5b [<c050f0c8>] cfq_cic_free+0x15/0x17 [<c050f128>] cic_free_func+0x5e/0x64 [<c050ea90>] __call_for_each_cic+0x23/0x2e [<c050eaad>] cfq_free_io_context+0x12/0x14 [<c050978c>] put_io_context+0x4b/0x66 [<c050f00a>] cfq_active_ioq_reset+0x21/0x39 [<c0511044>] elv_reset_active_ioq+0x2b/0x3e [<c0512ecf>] __elv_ioq_slice_expired+0x238/0x26a [<c0512f1f>] elv_ioq_slice_expired+0x1e/0x20 [<c0513860>] elv_ioq_request_add+0x366/0x393 [<c05048cd>] elv_insert+0x114/0x1a2 [<c05049ec>] __elv_add_request+0x91/0x96 [<c0507a00>] __make_request+0x365/0x397 [<c050635a>] generic_make_request+0x342/0x3ce [<c0507b21>] submit_bio+0xef/0xfa [<c04bf495>] submit_bh+0xe3/0x102 [<c04c04b0>] ll_rw_block+0xbe/0xf7 [<f80c35ba>] ext3_bread+0x39/0x79 [ext3] [<f80c5643>] dx_probe+0x2f/0x298 [ext3] [<f80c5956>] ext3_find_entry+0xaa/0x573 [ext3] [<f80c739e>] ext3_lookup+0x31/0xbe [ext3] [<c04abf7c>] do_lookup+0xbc/0x159 [<c04ad7e8>] __link_path_walk+0x6e9/0xa71 [<c04adbae>] path_walk+0x3e/0x77 [<c04add0e>] do_path_lookup+0xeb/0x105 [<c04ae584>] user_path_at+0x41/0x6c [<c04a8301>] vfs_fstatat+0x32/0x59 [<c04a8417>] vfs_stat+0x18/0x1a [<c04a8432>] sys_stat64+0x19/0x2d [<c0402a68>] sysenter_do_call+0x12/0x36 [<ffffffff>] 0xffffffff
-> (&iocg->lock){+.+...} ops: 3 {
HARDIRQ-ON-W at:
[<c044b840>] mark_held_locks+0x3d/0x58
[<c044b963>] trace_hardirqs_on_caller+0x108/0x14c
[<c044b9b2>] trace_hardirqs_on+0xb/0xd
[<c0630883>] _spin_unlock_irq+0x27/0x47
[<c0513baa>] iocg_destroy+0xbc/0x118
[<c045a16a>] cgroup_diput+0x4b/0xa7
[<c04b1dbb>] dentry_iput+0x78/0x9c
[<c04b1e82>] d_kill+0x21/0x3b
[<c04b2f2a>] dput+0xf3/0xfc
[<c04ae226>] do_rmdir+0x9a/0xc8
[<c04ae29d>] sys_rmdir+0x15/0x17
[<c0402a68>] sysenter_do_call+0x12/0x36
[<ffffffff>] 0xffffffff
SOFTIRQ-ON-W at:
[<c044b840>] mark_held_locks+0x3d/0x58
[<c044b97c>] trace_hardirqs_on_caller+0x121/0x14c
[<c044b9b2>] trace_hardirqs_on+0xb/0xd
[<c0630883>] _spin_unlock_irq+0x27/0x47
[<c0513baa>] iocg_destroy+0xbc/0x118
[<c045a16a>] cgroup_diput+0x4b/0xa7
[<c04b1dbb>] dentry_iput+0x78/0x9c
[<c04b1e82>] d_kill+0x21/0x3b
[<c04b2f2a>] dput+0xf3/0xfc
[<c04ae226>] do_rmdir+0x9a/0xc8
[<c04ae29d>] sys_rmdir+0x15/0x17
[<c0402a68>] sysenter_do_call+0x12/0x36
[<ffffffff>] 0xffffffff
INITIAL USE at:
[<c044dad5>] __lock_acquire+0x58c/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c06304ea>] _spin_lock_irq+0x30/0x3f
[<c05119bd>] io_alloc_root_group+0x104/0x155
[<c05133cb>] elv_init_fq_data+0x32/0xe0
[<c0504317>] elevator_alloc+0x150/0x170
[<c0505393>] elevator_init+0x9d/0x100
[<c0507088>] blk_init_queue_node+0xc4/0xf7
[<c05070cb>] blk_init_queue+0x10/0x12
[<f81060fd>] __scsi_alloc_queue+0x1c/0xba [scsi_mod]
[<f81061b0>] scsi_alloc_queue+0x15/0x4e [scsi_mod]
[<f810803d>] scsi_alloc_sdev+0x154/0x1f5 [scsi_mod]
[<f8108387>] scsi_probe_and_add_lun+0x123/0xb5b [scsi_mod]
[<f8109847>] __scsi_add_device+0x8a/0xb0 [scsi_mod]
[<f816ad14>] ata_scsi_scan_host+0x77/0x141 [libata]
[<f816903f>] async_port_probe+0xa0/0xa9 [libata]
[<c044341f>] async_thread+0xe9/0x1c9
[<c043e204>] kthread+0x4a/0x72
[<c04034e7>] kernel_thread_helper+0x7/0x10
[<ffffffff>] 0xffffffff
}
... key at: [<c0c5ebd8>] __key.29462+0x0/0x8
... acquired at:
[<c044d243>] validate_chain+0x8a8/0xbae
[<c044dbfd>] __lock_acquire+0x6b4/0x73e
[<c044dd36>] lock_acquire+0xaf/0xcc
[<c063056b>] _spin_lock_irqsave+0x33/0x43
[<c0510f6f>] io_group_chain_link+0x5c/0x106
[<c0511ba7>] io_find_alloc_group+0x54/0x60
[<c0511c11>] io_get_io_group_bio+0x5e/0x89
[<c0511cc3>] io_group_get_request_list+0x12/0x21
[<c0507485>] get_request_wait+0x124/0x15d
[<c050797e>] __make_request+0x2e3/0x397
[<c050635a>] generic_make_request+0x342/0x3ce
[<c0507b21>] submit_bio+0xef/0xfa
[<c04c6c4e>] mpage_bio_submit+0x21/0x26
[<c04c7b7f>] mpage_readpages+0xa3/0xad
[<f80c1ea8>] ext3_readpages+0x19/0x1b [ext3]
[<c048275e>] __do_page_cache_readahead+0xfd/0x166
[<c048294a>] ondemand_readahead+0x10a/0x118
[<c04829db>] page_cache_sync_readahead+0x1b/0x20
[<c047cf37>] generic_file_aio_read+0x226/0x545
[<c04a4cf6>] do_sync_read+0xb0/0xee
[<c04a54b0>] vfs_read+0x8f/0x136
[<c04a8d7c>] kernel_read+0x39/0x4b
[<c04a8e69>] prepare_binprm+0xdb/0xe3
[<c04a9ca8>] do_execve+0x175/0x2cc
[<c04015c0>] sys_execve+0x2b/0x54
[<c0402a68>] sysenter_do_call+0x12/0x36
[<ffffffff>] 0xffffffff
stack backtrace:
Pid: 2186, comm: rmdir Not tainted 2.6.30-rc4-io #6
Call Trace:
[<c044b1ac>] print_irq_inversion_bug+0x13b/0x147
[<c044c3e5>] check_usage_backwards+0x7d/0x86
[<c044b5ec>] mark_lock+0x2d3/0x4ea
[<c044c368>] ? check_usage_backwards+0x0/0x86
[<c044b840>] mark_held_locks+0x3d/0x58
[<c0630883>] ? _spin_unlock_irq+0x27/0x47
[<c044b97c>] trace_hardirqs_on_caller+0x121/0x14c
[<c044b9b2>] trace_hardirqs_on+0xb/0xd
[<c0630883>] _spin_unlock_irq+0x27/0x47
[<c0513baa>] iocg_destroy+0xbc/0x118
[<c045a16a>] cgroup_diput+0x4b/0xa7
[<c04b1dbb>] dentry_iput+0x78/0x9c
[<c04b1e82>] d_kill+0x21/0x3b
[<c04b2f2a>] dput+0xf3/0xfc
[<c04ae226>] do_rmdir+0x9a/0xc8
[<c04029b1>] ? resume_userspace+0x11/0x28
[<c051aa14>] ? trace_hardirqs_on_thunk+0xc/0x10
[<c0402b34>] ? restore_nocheck_notrace+0x0/0xe
[<c06324a0>] ? do_page_fault+0x0/0x2fd
[<c044b97c>] ? trace_hardirqs_on_caller+0x121/0x14c
[<c04ae29d>] sys_rmdir+0x15/0x17
[<c0402a68>] sysenter_do_call+0x12/0x36_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers
Quoted 8 lines · Vivek Goyal
Hi Gui,
Thanks for the report. I use
cgroup_path()for debugging. I guess thatcgroup_path()was passed null cgrp pointer that's why it crashed.If yes, then it is strange though. I call
cgroup_path()only after grabbing a refenrece to css object. (I am assuming that if I have a valid reference to css object thencss->cgrpcan't be null).
I think so too...
Quoted 23 lines
Anyway, can you please try out following patch and see if it fixes your crash.
--- block/elevator-fq.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)Index:
linux11/block/elevator-fq.c
===================================================================block/elevator-fq.c +3
-0@@ -125,6 +125,9 @@ static void io_group_path(struct io_grouunsigned short id = iog->iocg_id;struct cgroup_subsys_state *css; +/* For error case */ + buf[0 ] ='\0' ; +rcu_read_lock ();if (!id)
block/elevator-fq.c static void io_group_path(struct io_grou · line 143
Quoted 5 lines
block/elevator-fq.c +1
-1@@ -137,15 +140,12 @@ static void io_group_path(struct io_grouif (!css_tryget (css)) goto out; -cgroup_path (css->cgroup, buf, buflen); +if (css->cgroup)
According to CR2, when kernel crashing, css->cgroup equals 0x00000100. So i guess this patch won't fix this issue.
Quoted 13 lines
block/elevator-fq.c +1
-4+cgroup_path (css->cgroup, buf, buflen);css_put (css); - -rcu_read_unlock (); -return ; out:rcu_read_unlock (); - buf[0 ] ='\0' ; return; }#endif
Quoted 2 lines
BTW, I tried following equivalent script and I can't see the crash on my system. Are you able to hit it regularly?
yes, it's 50% chance that i can reproduce it.
i'v attached the rwio source code.
Quoted 10 lines
Instead of killing the tasks I also tried moving the tasks into root cgroup and then deleting test1 and test2 groups, that also did not produce any crash. (Hit a different bug though after 5-6 attempts :-)
As I mentioned in the patchset, currently we do have issues with group refcounting and cgroup/group going away. Hopefully in next version they all should be fixed up. But still, it is nice to hear back...
Signature · 2 lines
Regards Gui Jianfeng
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers
Quoted 8 lines · Vivek Goyal
Hi Gui,
Thanks for the report. I use
cgroup_path()for debugging. I guess thatcgroup_path()was passed null cgrp pointer that's why it crashed.If yes, then it is strange though. I call
cgroup_path()only after grabbing a refenrece to css object. (I am assuming that if I have a valid reference to css object thencss->cgrpcan't be null).
I think so too...
Quoted 23 lines
Anyway, can you please try out following patch and see if it fixes your crash.
--- block/elevator-fq.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)Index:
linux11/block/elevator-fq.c
===================================================================block/elevator-fq.c +3
-0@@ -125,6 +125,9 @@ static void io_group_path(struct io_grouunsigned short id = iog->iocg_id;struct cgroup_subsys_state *css; +/* For error case */ + buf[0 ] ='\0' ; +rcu_read_lock ();if (!id)
block/elevator-fq.c static void io_group_path(struct io_grou · line 143
Quoted 5 lines
block/elevator-fq.c +1
-1@@ -137,15 +140,12 @@ static void io_group_path(struct io_grouif (!css_tryget (css)) goto out; -cgroup_path (css->cgroup, buf, buflen); +if (css->cgroup)
According to CR2, when kernel crashing, css->cgroup equals 0x00000100. So i guess this patch won't fix this issue.
Quoted 13 lines
block/elevator-fq.c +1
-4+cgroup_path (css->cgroup, buf, buflen);css_put (css); - -rcu_read_unlock (); -return ; out:rcu_read_unlock (); - buf[0 ] ='\0' ; return; }#endif
Quoted 2 lines
BTW, I tried following equivalent script and I can't see the crash on my system. Are you able to hit it regularly?
yes, it's 50% chance that i can reproduce it.
i'v attached the rwio source code.
Quoted 10 lines
Instead of killing the tasks I also tried moving the tasks into root cgroup and then deleting test1 and test2 groups, that also did not produce any crash. (Hit a different bug though after 5-6 attempts :-)
As I mentioned in the patchset, currently we do have issues with group refcounting and cgroup/group going away. Hopefully in next version they all should be fixed up. But still, it is nice to hear back...
Signature · 2 lines
Regards Gui Jianfeng
PATCH 1/1 io-controller: Add io group reference handling for request
In reply to Vivek GoyalHi Vivek,
This patch adds io group reference handling when allocating and removing a request.
--- elevator-fq.c | 15 ++++++++++++++- elevator-fq.h | 5 +++++ elevator.c | 2 ++ 3 files changed, 21 insertions(+), 1 deletion(-)
block/elevator-fq.c +14 -1
@@ -1968,11 +1968,24 @@ void elv_fq_set_request_io_group(struct request_queue *q, struct request *rq,spin_unlock_irqrestore (q->queue_lock, flags);BUG_ON (!iog); -/* Store iog in rq. TODO: take care of referencing */ +elv_get_iog (iog); rq->iog = iog; }/* +* This request has been serviced. Clean up iog info and drop the reference. +*/ +void elv_fq_unset_request_io_group (struct request *rq) +{ +struct io_group *iog = rq->iog; + +if (iog) { + rq->iog =NULL ; +elv_put_iog (iog); + } +} + +/* * Find/Create the io queue the rq should go in. This is an optimization * for the io schedulers (noop, deadline and AS) which maintain only single * io queue per cgroup. In this case common layer can just maintain a
block/elevator-fq.h +5 -0
@@ -512,6 +512,7 @@ static inline struct io_group *ioq_to_io_group(struct io_queue *ioq)extern int io_group_allow_merge (struct request *rq,struct bio *bio);extern void elv_fq_set_request_io_group (struct request_queue *q,struct request *rq,struct bio *bio); +extern void elv_fq_unset_request_io_group (struct request *rq);static inline bfq_weight_t iog_weight (struct io_group *iog) {return iog->entity.weight;
@@ -571,6 +572,10 @@ static inline void elv_fq_set_request_io_group(struct request_queue *q, { } +static inline void elv_fq_unset_request_io_group (struct request *rq) +{ +} +static inline bfq_weight_t iog_weight (struct io_group *iog) {/* Just root group is present and weight is immaterial. */
block/elevator.c +2 -0
@@ -992,6 +992,8 @@ void elv_put_request(struct request_queue *q, struct request *rq) {struct elevator_queue *e = q->elevator; +elv_fq_unset_request_io_group (rq); +/* * Optimization for noop, deadline and AS which maintain only single * ioq per io group
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 5 lines · Gui Jianfeng
Hi Vivek,
This patch adds io group reference handling when allocating and removing a request.
Hi Gui,
Thanks for the patch. We were thinking that requests can take a reference on io queues and io queues can take a reference on io groups. That should make sure that io groups don't go away as long as active requests are present.
But there seems to be a small window while allocating the new request where request gets allocated from a group first and then later it is mapped to that group and queue is created. IOW, in get_request_wait(), we allocate a request from a particular group and set rq->rl, then drop the queue lock and later call elv_set_request() which again maps the request to the group saves rq->iog and creates new queue. This window is troublesome because request can be mapped to a particular group at the time of allocation and during set_request() it can go to a different group as queue lock was dropped and group might have disappeared.
In this case probably it might make sense that request also takes a reference on groups. At the same time it looks too much that request takes a reference on queue as well as group object. Ideas are welcome on how to handle it...
Thanks
Vivek
Quoted 74 lines
Signed-off-by Gui Jianfeng <[email protected]>--- elevator-fq.c | 15 ++++++++++++++- elevator-fq.h | 5 +++++ elevator.c | 2 ++ 3 files changed, 21 insertions(+), 1 deletion(-)block/elevator-fq.c +14
-1@@ -1968,11 +1968,24 @@ void elv_fq_set_request_io_group(struct request_queue *q, struct request *rq,spin_unlock_irqrestore (q->queue_lock, flags);BUG_ON (!iog); -/* Store iog in rq. TODO: take care of referencing */ +elv_get_iog (iog); rq->iog = iog; }/* +* This request has been serviced. Clean up iog info and drop the reference. +*/ +void elv_fq_unset_request_io_group (struct request *rq) +{ +struct io_group *iog = rq->iog; + +if (iog) { + rq->iog =NULL ; +elv_put_iog (iog); + } +} + +/* * Find/Create the io queue the rq should go in. This is an optimization * for the io schedulers (noop, deadline and AS) which maintain only single * io queue per cgroup. In this case common layer can just maintain a block/elevator-fq.h +5
-0@@ -512,6 +512,7 @@ static inline struct io_group *ioq_to_io_group(struct io_queue *ioq)extern int io_group_allow_merge (struct request *rq,struct bio *bio);extern void elv_fq_set_request_io_group (struct request_queue *q,struct request *rq,struct bio *bio); +extern void elv_fq_unset_request_io_group (struct request *rq);static inline bfq_weight_t iog_weight (struct io_group *iog) {return iog->entity.weight;@@ -571,6 +572,10 @@ static inline void elv_fq_set_request_io_group(struct request_queue *q, { } +static inline void elv_fq_unset_request_io_group (struct request *rq) +{ +} +static inline bfq_weight_t iog_weight (struct io_group *iog) {/* Just root group is present and weight is immaterial. */ block/elevator.c +2
-0@@ -992,6 +992,8 @@ void elv_put_request(struct request_queue *q, struct request *rq) {struct elevator_queue *e = q->elevator; +elv_fq_unset_request_io_group (rq); +/* * Optimization for noop, deadline and AS which maintain only single * ioq per io group
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 24 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Hi Vivek,
This patch adds io group reference handling when allocating and removing a request.
Hi Gui,
Thanks for the patch. We were thinking that requests can take a reference on io queues and io queues can take a reference on io groups. That should make sure that io groups don't go away as long as active requests are present.
But there seems to be a small window while allocating the new request where request gets allocated from a group first and then later it is mapped to that group and queue is created. IOW, in
get_request_wait(), we allocate a request from a particular group and setrq->rl, then drop the queue lock and later callelv_set_request()which again maps the request to the group savesrq->iogand creates new queue. This window is troublesome because request can be mapped to a particular group at the time of allocation and duringset_request()it can go to a different group as queue lock was dropped and group might have disappeared.
Quoted 4 lines
In this case probably it might make sense that request also takes a reference on groups. At the same time it looks too much that request takes a reference on queue as well as group object. Ideas are welcome on how to handle it...
IMHO a request being allocated on the wrong cgroup should not be a big problem as such. All it means is that the request descriptor was accounted to the wrong cgroup in this particular corner case. Please correct me if I am wrong.
We can also get rid of rq->iog pointer too. What that means is that request is associated with ioq (rq->ioq), and we can use ioq_to_io_group() function to get the io_group. So the request would only be indirectly associated with an io_group i.e. the request is associated with an io_queue and the io_group for the request is the io_group associated with io_queue. Do you see any problems with that approach?
Thanks.
--
Nauman
Quoted 79 lines
Thanks
VivekQuoted 74 lines
Signed-off-by Gui Jianfeng <[email protected]>---
elevator-fq.c | 15 ++++++++++++++-
elevator-fq.h | 5 +++++
elevator.c | 2 ++3 files changed, 21 insertions(+), 1 deletion(-)block/elevator-fq.c +14
-1@@ -1968,11 +1968,24 @@ void elv_fq_set_request_io_group(struct request_queue *q, struct request *rq, spin_unlock_irqrestore(q->queue_lock, flags); BUG_ON(!iog); - /* Store iog in rq. TODO: take care of referencing */ + elv_get_iog(iog); rq->iog = iog; } /* +* This request has been serviced. Clean up iog info and drop the reference. +*/ +void elv_fq_unset_request_io_group (struct request *rq) +{ + struct io_group *iog = rq->iog; + + if (iog) { + rq->iog = NULL; + elv_put_iog(iog); + } +} + +/* * Find/Create the io queue the rq should go in. This is an optimization * for the io schedulers (noop, deadline and AS) which maintain only single * io queue per cgroup. In this case common layer can just maintain a block/elevator-fq.h +5
-0@@ -512,6 +512,7 @@ static inline struct io_group *ioq_to_io_group(struct io_queue *ioq) extern int io_group_allow_merge(struct request *rq, struct bio *bio); extern void elv_fq_set_request_io_group(struct request_queue *q, struct request *rq, struct bio *bio); +extern void elv_fq_unset_request_io_group (struct request *rq); static inline bfq_weight_t iog_weight(struct io_group *iog) { return iog->entity.weight;@@ -571,6 +572,10 @@ static inline void elv_fq_set_request_io_group(struct request_queue *q, { } +static inline void elv_fq_unset_request_io_group (struct request *rq) +{ +} + static inline bfq_weight_t iog_weight(struct io_group *iog) { /* Just root group is present and weight is immaterial. */block/elevator.c +2
-0@@ -992,6 +992,8 @@ void elv_put_request(struct request_queue *q, struct request *rq) { struct elevator_queue *e = q->elevator; + elv_fq_unset_request_io_group(rq); + /** Optimization for noop, deadline and AS which maintain only single * ioq per io group
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 30 lines · Nauman Rafique
Quoted 24 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Hi Vivek,
This patch adds io group reference handling when allocating and removing a request.
Hi Gui,
Thanks for the patch. We were thinking that requests can take a reference on io queues and io queues can take a reference on io groups. That should make sure that io groups don't go away as long as active requests are present.
But there seems to be a small window while allocating the new request where request gets allocated from a group first and then later it is mapped to that group and queue is created. IOW, in
get_request_wait(), we allocate a request from a particular group and setrq->rl, then drop the queue lock and later callelv_set_request()which again maps the request to the group savesrq->iogand creates new queue. This window is troublesome because request can be mapped to a particular group at the time of allocation and duringset_request()it can go to a different group as queue lock was dropped and group might have disappeared.Quoted 4 lines
In this case probably it might make sense that request also takes a reference on groups. At the same time it looks too much that request takes a reference on queue as well as group object. Ideas are welcome on how to handle it...
Quoted 4 lines
IMHO a request being allocated on the wrong cgroup should not be a big problem as such. All it means is that the request descriptor was accounted to the wrong cgroup in this particular corner case. Please correct me if I am wrong.
I think you are right. We just need to be little careful while freeing the request that associated request list might have gone away (rq->rl).
Or we probably can think of getting rid of (rq->rl) also and while freeing request determine io queue and group from rq->ioq. But somehow I remember that I had to introduce rq->rl otherwise I was running into issues of request being mapped to different groups at different point of time hence different request list etc. Will check again..
Quoted 8 lines
We can also get rid of
rq->iogpointer too. What that means is that request is associated with ioq (rq->ioq), and we can useioq_to_io_group()function to get theio_group. So the request would only be indirectly associated with anio_groupi.e. the request is associated with anio_queueand theio_groupfor the request is theio_groupassociated withio_queue. Do you see any problems with that approach?
Looks like this is also doable. Good idea. Can't think of why can't we get rid of rq->iog and manage with rq->ioq. There are only 1-2 places where ioq is not setup yet and rq has been mapped to the group. There we shall have to carry group information or carry bio information and map it again to get group info.
Will try to implement it and see how does it go.
Thanks
Vivek
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 53 lines · Vivek Goyal
Quoted 30 lines · Nauman Rafique
Quoted 24 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Hi Vivek,
This patch adds io group reference handling when allocating and removing a request.
Hi Gui,
Thanks for the patch. We were thinking that requests can take a reference on io queues and io queues can take a reference on io groups. That should make sure that io groups don't go away as long as active requests are present.
But there seems to be a small window while allocating the new request where request gets allocated from a group first and then later it is mapped to that group and queue is created. IOW, in
get_request_wait(), we allocate a request from a particular group and setrq->rl, then drop the queue lock and later callelv_set_request()which again maps the request to the group savesrq->iogand creates new queue. This window is troublesome because request can be mapped to a particular group at the time of allocation and duringset_request()it can go to a different group as queue lock was dropped and group might have disappeared.Quoted 4 lines
In this case probably it might make sense that request also takes a reference on groups. At the same time it looks too much that request takes a reference on queue as well as group object. Ideas are welcome on how to handle it...
Quoted 4 lines
IMHO a request being allocated on the wrong cgroup should not be a big problem as such. All it means is that the request descriptor was accounted to the wrong cgroup in this particular corner case. Please correct me if I am wrong.
I think you are right. We just need to be little careful while freeing the request that associated request list might have gone away (
rq->rl).Or we probably can think of getting rid of (
rq->rl) also and while freeing request determine io queue and group fromrq->ioq. But somehow I remember that I had to introducerq->rlotherwise I was running into issues of request being mapped to different groups at different point of time hence different request list etc. Will check again..Quoted 8 lines
We can also get rid of
rq->iogpointer too. What that means is that request is associated with ioq (rq->ioq), and we can useioq_to_io_group()function to get theio_group. So the request would only be indirectly associated with anio_groupi.e. the request is associated with anio_queueand theio_groupfor the request is theio_groupassociated withio_queue. Do you see any problems with that approach?
Quoted 7 lines
Looks like this is also doable. Good idea. Can't think of why can't we get rid of
rq->iogand manage withrq->ioq. There are only 1-2 places where ioq is not setup yet and rq has been mapped to the group. There we shall have to carry group information or carry bio information and map it again to get group info.Will try to implement it and see how does it go.
I tried it, and it seems to work. I passed io_group around as function arguments, and return values before ioq was set.
Quoted 4 lines
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 53 lines · Vivek Goyal
Quoted 30 lines · Nauman Rafique
Quoted 24 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Hi Vivek,
This patch adds io group reference handling when allocating and removing a request.
Hi Gui,
Thanks for the patch. We were thinking that requests can take a reference on io queues and io queues can take a reference on io groups. That should make sure that io groups don't go away as long as active requests are present.
But there seems to be a small window while allocating the new request where request gets allocated from a group first and then later it is mapped to that group and queue is created. IOW, in
get_request_wait(), we allocate a request from a particular group and setrq->rl, then drop the queue lock and later callelv_set_request()which again maps the request to the group savesrq->iogand creates new queue. This window is troublesome because request can be mapped to a particular group at the time of allocation and duringset_request()it can go to a different group as queue lock was dropped and group might have disappeared.Quoted 4 lines
In this case probably it might make sense that request also takes a reference on groups. At the same time it looks too much that request takes a reference on queue as well as group object. Ideas are welcome on how to handle it...
Quoted 4 lines
IMHO a request being allocated on the wrong cgroup should not be a big problem as such. All it means is that the request descriptor was accounted to the wrong cgroup in this particular corner case. Please correct me if I am wrong.
I think you are right. We just need to be little careful while freeing the request that associated request list might have gone away (
rq->rl).Or we probably can think of getting rid of (
rq->rl) also and while freeing request determine io queue and group fromrq->ioq. But somehow I remember that I had to introducerq->rlotherwise I was running into issues of request being mapped to different groups at different point of time hence different request list etc. Will check again..Quoted 8 lines
We can also get rid of
rq->iogpointer too. What that means is that request is associated with ioq (rq->ioq), and we can useioq_to_io_group()function to get theio_group. So the request would only be indirectly associated with anio_groupi.e. the request is associated with anio_queueand theio_groupfor the request is theio_groupassociated withio_queue. Do you see any problems with that approach?
Quoted 7 lines
Looks like this is also doable. Good idea. Can't think of why can't we get rid of
rq->iogand manage withrq->ioq. There are only 1-2 places where ioq is not setup yet and rq has been mapped to the group. There we shall have to carry group information or carry bio information and map it again to get group info.Will try to implement it and see how does it go.
I tried it, and it seems to work. I passed io_group around as function arguments, and return values before ioq was set.
Quoted 4 lines
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): e98e18940905081041r386e52a5q5a2b1f13f1e8c634-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 30 lines · Nauman Rafique
Quoted 24 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Hi Vivek,
This patch adds io group reference handling when allocating and removing a request.
Hi Gui,
Thanks for the patch. We were thinking that requests can take a reference on io queues and io queues can take a reference on io groups. That should make sure that io groups don't go away as long as active requests are present.
But there seems to be a small window while allocating the new request where request gets allocated from a group first and then later it is mapped to that group and queue is created. IOW, in
get_request_wait(), we allocate a request from a particular group and setrq->rl, then drop the queue lock and later callelv_set_request()which again maps the request to the group savesrq->iogand creates new queue. This window is troublesome because request can be mapped to a particular group at the time of allocation and duringset_request()it can go to a different group as queue lock was dropped and group might have disappeared.Quoted 4 lines
In this case probably it might make sense that request also takes a reference on groups. At the same time it looks too much that request takes a reference on queue as well as group object. Ideas are welcome on how to handle it...
Quoted 4 lines
IMHO a request being allocated on the wrong cgroup should not be a big problem as such. All it means is that the request descriptor was accounted to the wrong cgroup in this particular corner case. Please correct me if I am wrong.
I think you are right. We just need to be little careful while freeing the request that associated request list might have gone away (rq->rl).
Or we probably can think of getting rid of (rq->rl) also and while freeing request determine io queue and group from rq->ioq. But somehow I remember that I had to introduce rq->rl otherwise I was running into issues of request being mapped to different groups at different point of time hence different request list etc. Will check again..
Quoted 8 lines
We can also get rid of
rq->iogpointer too. What that means is that request is associated with ioq (rq->ioq), and we can useioq_to_io_group()function to get theio_group. So the request would only be indirectly associated with anio_groupi.e. the request is associated with anio_queueand theio_groupfor the request is theio_groupassociated withio_queue. Do you see any problems with that approach?
Looks like this is also doable. Good idea. Can't think of why can't we get rid of rq->iog and manage with rq->ioq. There are only 1-2 places where ioq is not setup yet and rq has been mapped to the group. There we shall have to carry group information or carry bio information and map it again to get group info.
Will try to implement it and see how does it go.
Thanks
Vivek
Re: [PATCH] io-controller: Add io group reference handling for request
In reply to a message not in the archiveQuoted 34 lines · Nauman Rafique
Quoted 23 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Hi Vivek,
This patch adds io group reference handling when allocating and removing a request.
Hi Gui,
Thanks for the patch. We were thinking that requests can take a reference on io queues and io queues can take a reference on io groups. That should make sure that io groups don't go away as long as active requests are present.
But there seems to be a small window while allocating the new request where request gets allocated from a group first and then later it is mapped to that group and queue is created. IOW, in
get_request_wait(), we allocate a request from a particular group and setrq->rl, then drop the queue lock and later callelv_set_request()which again maps the request to the group savesrq->iogand creates new queue. This window is troublesome because request can be mapped to a particular group at the time of allocation and duringset_request()it can go to a different group as queue lock was dropped and group might have disappeared.Quoted 4 lines
In this case probably it might make sense that request also takes a reference on groups. At the same time it looks too much that request takes a reference on queue as well as group object. Ideas are welcome on how to handle it...
IMHO a request being allocated on the wrong cgroup should not be a big problem as such. All it means is that the request descriptor was accounted to the wrong cgroup in this particular corner case. Please correct me if I am wrong.
Quoted 7 lines
We can also get rid of
rq->iogpointer too. What that means is that request is associated with ioq (rq->ioq), and we can useioq_to_io_group()function to get theio_group. So the request would only be indirectly associated with anio_groupi.e. the request is associated with anio_queueand theio_groupfor the request is theio_groupassociated withio_queue. Do you see any problems with that approach?
That sounds reasonable to get rid of rq->iog, and rq->rl is also dead. Hope to see the patch soon. ;)
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] io-controller: Add io group reference handling for request
In reply to Nauman RafiqueQuoted 34 lines · Nauman Rafique
Quoted 23 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Hi Vivek,
This patch adds io group reference handling when allocating and removing a request.
Hi Gui,
Thanks for the patch. We were thinking that requests can take a reference on io queues and io queues can take a reference on io groups. That should make sure that io groups don't go away as long as active requests are present.
But there seems to be a small window while allocating the new request where request gets allocated from a group first and then later it is mapped to that group and queue is created. IOW, in
get_request_wait(), we allocate a request from a particular group and setrq->rl, then drop the queue lock and later callelv_set_request()which again maps the request to the group savesrq->iogand creates new queue. This window is troublesome because request can be mapped to a particular group at the time of allocation and duringset_request()it can go to a different group as queue lock was dropped and group might have disappeared.Quoted 4 lines
In this case probably it might make sense that request also takes a reference on groups. At the same time it looks too much that request takes a reference on queue as well as group object. Ideas are welcome on how to handle it...
IMHO a request being allocated on the wrong cgroup should not be a big problem as such. All it means is that the request descriptor was accounted to the wrong cgroup in this particular corner case. Please correct me if I am wrong.
Quoted 7 lines
We can also get rid of
rq->iogpointer too. What that means is that request is associated with ioq (rq->ioq), and we can useioq_to_io_group()function to get theio_group. So the request would only be indirectly associated with anio_groupi.e. the request is associated with anio_queueand theio_groupfor the request is theio_groupassociated withio_queue. Do you see any problems with that approach?
That sounds reasonable to get rid of rq->iog, and rq->rl is also dead. Hope to see the patch soon. ;)
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 43 lines · Gui Jianfeng
Quoted 34 lines · Nauman Rafique
Quoted 23 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Hi Vivek,
This patch adds io group reference handling when allocating and removing a request.
Hi Gui,
Thanks for the patch. We were thinking that requests can take a reference on io queues and io queues can take a reference on io groups. That should make sure that io groups don't go away as long as active requests are present.
But there seems to be a small window while allocating the new request where request gets allocated from a group first and then later it is mapped to that group and queue is created. IOW, in
get_request_wait(), we allocate a request from a particular group and setrq->rl, then drop the queue lock and later callelv_set_request()which again maps the request to the group savesrq->iogand creates new queue. This window is troublesome because request can be mapped to a particular group at the time of allocation and duringset_request()it can go to a different group as queue lock was dropped and group might have disappeared.Quoted 4 lines
In this case probably it might make sense that request also takes a reference on groups. At the same time it looks too much that request takes a reference on queue as well as group object. Ideas are welcome on how to handle it...
IMHO a request being allocated on the wrong cgroup should not be a big problem as such. All it means is that the request descriptor was accounted to the wrong cgroup in this particular corner case. Please correct me if I am wrong.
Quoted 7 lines
We can also get rid of
rq->iogpointer too. What that means is that request is associated with ioq (rq->ioq), and we can useioq_to_io_group()function to get theio_group. So the request would only be indirectly associated with anio_groupi.e. the request is associated with anio_queueand theio_groupfor the request is theio_groupassociated withio_queue. Do you see any problems with that approach?
Quoted 3 lines
That sounds reasonable to get rid of
rq->iog, andrq->rlis also dead. Hope to see the patch soon. ;)
Ok, here is the patch which gets rid of rq->iog and rq->rl fields. Good to see some code and data structures trimming. It seems to be working fine for me.
o Get rid of rq->iog field and rq->rl fields. request descriptor stores the pointer the the queue it belongs to (rq->ioq) and from the io queue one can determine the group queue belongs to hence request belongs to. Thanks to Nauman for the idea.
o There are couple of places where rq->ioq information is not present yet as request and queue are being setup. In those places "bio" is passed around as function argument to determine the group rq will go into. I did not pass "iog" as function argument becuase when memory is scarce, we can release queue lock and sleep to wait for memory to become available and once we wake up, it is possible that io group is gone. Passing bio around helps that one shall have to remap bio to right group after waking up.
o Got rid of io_lookup_io_group_current() function and merged it with io_get_io_group() to also take care of looking for group using current task info and not from bio.
--- block/blk-core.c | 28 +++++++++------- block/cfq-iosched.c | 40 ++++++++++++++++-------- block/elevator-fq.c | 78 ++++++++++++++++++----------------------------- block/elevator-fq.h | 29 +++-------------- block/elevator.c | 6 +-- include/linux/blkdev.h | 16 ++++----- include/linux/elevator.h | 2 - 7 files changed, 91 insertions(+), 108 deletions(-)
include/linux/blkdev.h +8 -8
@@ -279,12 +279,6 @@ struct request {#ifdef CONFIG_ELV_FAIR_QUEUING /* io queue request belongs to */ struct io_queue *ioq; - -#ifdef CONFIG_GROUP_IOSCHED -/* io group request belongs to */ -struct io_group *iog; -struct request_list *rl; -#endif /* GROUP_IOSCHED */ #endif /* ELV_FAIR_QUEUING */ };
@@ -828,9 +822,15 @@ static inline struct request_list *rq_rl struct request *rq) {#ifdef CONFIG_GROUP_IOSCHED -return rq->rl; +struct io_group *iog; + +BUG_ON (!rq->ioq); + iog =ioq_to_io_group (rq->ioq); +BUG_ON (!iog); + +return &iog->rl;#else -return blk_get_request_list (q,NULL ); +return &q->rq;#endif }
block/elevator-fq.c +31 -47
164 lines not on this page · raw message
include/linux/elevator.h +1 -1
@@ -23,7 +23,7 @@ typedef struct request *(elevator_requestypedef void (elevator_completed_req_fn ) (struct request_queue *,struct request *);typedef int (elevator_may_queue_fn ) (struct request_queue *,int ); -typedef int (elevator_set_req_fn ) (struct request_queue *,struct request *,gfp_t ); +typedef int (elevator_set_req_fn ) (struct request_queue *,struct request *,struct bio *bio,gfp_t );typedef void (elevator_put_req_fn ) (struct request *);typedef void (elevator_activate_req_fn ) (struct request_queue *,struct request *);typedef void (elevator_deactivate_req_fn ) (struct request_queue *,struct request *);
block/cfq-iosched.c +28 -12
@@ -161,7 +161,7 @@ CFQ_CFQQ_FNS(coop); blk_add_trace_msg((cfqd)->queue, "cfq " fmt, ##args) static void cfq_dispatch_insert(struct request_queue *, struct request *); -static struct cfq_queue *cfq_get_queue(struct cfq_data *, struct io_group *iog, +static struct cfq_queue *cfq_get_queue(struct cfq_data *, struct bio *bio, int, struct io_context *, gfp_t); static struct cfq_io_context *cfq_cic_lookup(struct cfq_data *, struct io_context *);
@@ -196,7 +196,7 @@ static struct cfq_queue *cic_bio_to_cfqq* async bio tracking is enabled and we are not caching * async queue pointer in cic. */ - iog =io_get_io_group_bio (cfqd->queue, bio,0 ); + iog =io_get_io_group (cfqd->queue, bio,0 ,0 );if (!iog) {/* * May be this is first rq/bio and io group has not
@@ -1242,7 +1242,6 @@ static void changed_ioprio(struct io_con cfqq = cic->cfqq[BLK_RW_ASYNC ];if (cfqq) { -struct io_group *iog =io_lookup_io_group_current (q);struct cfq_queue *new_cfqq;/*
@@ -1259,7 +1258,7 @@ static void changed_ioprio(struct io_con* comes? Keeping it for the time being because existing cfq * code allocates the new queue immediately upon prio change */ - new_cfqq =cfq_get_queue (cfqd, iog,BLK_RW_ASYNC , cic->ioc, + new_cfqq =cfq_get_queue (cfqd,NULL ,BLK_RW_ASYNC , cic->ioc,GFP_ATOMIC );if (new_cfqq)cic_set_cfqq (cic, new_cfqq,BLK_RW_ASYNC );
@@ -1295,7 +1294,7 @@ static void changed_cgroup(struct io_conspin_lock_irqsave (q->queue_lock, flags); - iog =io_lookup_io_group_current (q); + iog =io_get_io_group (q,NULL ,0 ,1 );if (async_cfqq !=NULL ) { __iog =cfqq_to_io_group (async_cfqq);
@@ -1332,14 +1331,25 @@ static void cfq_ioc_set_cgroup(struct io#endif /* CONFIG_IOSCHED_CFQ_HIER */ static struct cfq_queue * -cfq_find_alloc_queue (struct cfq_data *cfqd,struct io_group *iog,int is_sync, +cfq_find_alloc_queue (struct cfq_data *cfqd,struct bio *bio,int is_sync,struct io_context *ioc,gfp_t gfp_mask) {struct cfq_queue *cfqq, *new_cfqq =NULL ;struct cfq_io_context *cic;struct request_queue *q = cfqd->queue;struct io_queue *ioq =NULL , *new_ioq =NULL ; +struct io_group *iog =NULL ; retry: +/* +* Note: Finding the io group again in case io group disappeared +* during the time we dropped the queue lock and acquired it +* back. +*/ +if (bio) + iog =io_get_io_group (q, bio,1 ,0 ); +else + iog =io_get_io_group (q,NULL ,1 ,1 ); + cic =cfq_cic_lookup (cfqd, ioc);/* cic always exists here */ cfqq =cic_to_cfqq (cic, is_sync);
@@ -1449,13 +1459,19 @@ out: }static struct cfq_queue * -cfq_get_queue (struct cfq_data *cfqd,struct io_group *iog,int is_sync, -struct io_context *ioc,gfp_t gfp_mask) +cfq_get_queue (struct cfq_data *cfqd,struct bio *bio,int is_sync, +struct io_context *ioc,gfp_t gfp_mask) {const int ioprio =task_ioprio (ioc);const int ioprio_class =task_ioprio_class (ioc);struct cfq_queue *async_cfqq =NULL ;struct cfq_queue *cfqq =NULL ; +struct io_group *iog =NULL ; + +if (bio) + iog =io_get_io_group (cfqd->queue, bio,1 ,0 ); +else + iog =io_get_io_group (cfqd->queue,NULL ,1 ,1 );if (!is_sync) { async_cfqq = io_group_async_queue_prio(iog, ioprio_class,
@@ -1464,7 +1480,7 @@ cfq_get_queue(struct cfq_data *cfqd, str }if (!cfqq) { - cfqq =cfq_find_alloc_queue (cfqd, iog, is_sync, ioc, gfp_mask); + cfqq =cfq_find_alloc_queue (cfqd, bio, is_sync, ioc, gfp_mask);if (!cfqq)return NULL ; }
@@ -1889,7 +1905,8 @@ static void cfq_put_request(struct reque* Allocate cfq data structures associated with this request. */ static int -cfq_set_request (struct request_queue *q,struct request *rq,gfp_t gfp_mask) +cfq_set_request (struct request_queue *q,struct request *rq,struct bio *bio, +gfp_t gfp_mask) {struct cfq_data *cfqd = q->elevator->elevator_data;struct cfq_io_context *cic;
@@ -1909,8 +1926,7 @@ cfq_set_request(struct request_queue *q, cfqq =cic_to_cfqq (cic, is_sync);if (!cfqq) { - cfqq =cfq_get_queue (cfqd,rq_iog (q, rq), is_sync, cic->ioc, - gfp_mask); + cfqq =cfq_get_queue (cfqd, bio, is_sync, cic->ioc, gfp_mask);if (!cfqq) goto queue_fail;
block/elevator.c +2 -4
@@ -972,17 +972,15 @@ int elv_set_request(struct request_queue {struct elevator_queue *e = q->elevator; -elv_fq_set_request_io_group (q, rq, bio); -/* * Optimization for noop, deadline and AS which maintain only single * ioq per io group */ if (elv_iosched_single_ioq (e)) -return elv_fq_set_request_ioq (q, rq, gfp_mask); +return elv_fq_set_request_ioq (q, rq, bio, gfp_mask);if (e->ops->elevator_set_req_fn) -return e->ops->elevator_set_req_fn (q, rq, gfp_mask); +return e->ops->elevator_set_req_fn (q, rq, bio, gfp_mask); rq->elevator_private =NULL ;return 0 ;
block/elevator-fq.h +5 -24
@@ -510,15 +510,13 @@ static inline struct io_group *ioq_to_io#ifdef CONFIG_GROUP_IOSCHED extern int io_group_allow_merge (struct request *rq,struct bio *bio); -extern void elv_fq_set_request_io_group (struct request_queue *q, -struct request *rq,struct bio *bio);static inline bfq_weight_t iog_weight (struct io_group *iog) {return iog->entity.weight; }extern int elv_fq_set_request_ioq (struct request_queue *q,struct request *rq, -gfp_t gfp_mask); +struct bio *bio,gfp_t gfp_mask);extern void elv_fq_unset_request_ioq (struct request_queue *q,struct request *rq);extern struct io_queue *elv_lookup_ioq_current (struct request_queue *q);
@@ -545,12 +543,6 @@ static inline void io_group_set_ioq(stru iog->ioq = ioq; } -static inline struct io_group *rq_iog (struct request_queue *q, -struct request *rq) -{ -return rq->iog; -} -static inline void elv_get_iog (struct io_group *iog) {atomic_inc (&iog->ref);
@@ -566,10 +558,6 @@ static inline int io_group_allow_merge(s* separately. Hence in case of non-hierarchical setup, nothing todo. */ static inline void io_disconnect_groups (struct elevator_queue *e) {} -static inline void elv_fq_set_request_io_group (struct request_queue *q, -struct request *rq,struct bio *bio) -{ -}static inline bfq_weight_t iog_weight (struct io_group *iog) {
@@ -588,7 +576,7 @@ static inline void io_group_set_ioq(stru }static inline int elv_fq_set_request_ioq (struct request_queue *q, -struct request *rq,gfp_t gfp_mask) +struct request *rq,struct bio *bio,gfp_t gfp_mask) {return 0 ; }
@@ -613,8 +601,6 @@ static inline void elv_get_iog(struct iostatic inline void elv_put_iog (struct io_group *iog) { } -extern struct io_group *rq_iog (struct request_queue *q,struct request *rq); -#endif /* GROUP_IOSCHED */ /* Functions used by blksysfs.c */
@@ -670,8 +656,8 @@ extern void *io_group_async_queue_prio(sextern void io_group_set_async_queue (struct io_group *iog,int ioprio_class,int ioprio,struct io_queue *ioq);extern struct io_group *io_lookup_io_group_current (struct request_queue *q); -extern struct io_group *io_get_io_group_bio (struct request_queue *q, -struct bio *bio,int create); +extern struct io_group *io_get_io_group (struct request_queue *q, +struct bio *bio,int create,int curr);extern int elv_nr_busy_ioq (struct elevator_queue *e);extern int elv_nr_busy_rt_ioq (struct elevator_queue *e);extern struct io_queue *elv_alloc_ioq (struct request_queue *q,gfp_t gfp_mask);
@@ -725,18 +711,13 @@ static inline void *elv_fq_select_ioq(streturn NULL ; } -static inline void elv_fq_set_request_io_group (struct request_queue *q, -struct request *rq,struct bio *bio) -{ -} -static inline int io_group_allow_merge (struct request *rq,struct bio *bio) {return 1 ; }static inline int elv_fq_set_request_ioq (struct request_queue *q, -struct request *rq,gfp_t gfp_mask) +struct request *rq,struct bio *bio,gfp_t gfp_mask) {return 0 ; }
block/blk-core.c +16 -12
@@ -736,8 +736,22 @@ static void __freed_request(struct requestatic void freed_request (struct request_queue *q,int sync,int priv,struct request_list *rl) { -BUG_ON (!rl->count[sync]); - rl->count[sync]--; +/* There is a window during request allocation where request is +* mapped to one group but by the time a queue for the group is +* allocated, it is possible that original cgroup/io group has been +* deleted and now io queue is allocated in a different group (root) +* altogether. +* +* One solution to the problem is that rq should take io group +* reference. But it looks too much to do that to solve this issue. +* The only side affect to the hard to hit issue seems to be that +* we will try to decrement the rl->count for a request list which +* did not allocate that request. Chcek for rl->count going less than +* zero and do not decrement it if that's the case. +*/ + +if (rl->count[sync] >0 ) + rl->count[sync]--;BUG_ON (!q->rq_data.count[sync]); q->rq_data.count[sync]--;
@@ -841,16 +855,6 @@ static struct request *get_request(struc rq =blk_alloc_request (q, bio, rw_flags, priv, gfp_mask); -#ifdef CONFIG_GROUP_IOSCHED -if (rq) { -/* -* TODO. Implement group reference counting and take the -* reference to the group to make sure group hence request -* list does not go away till rq finishes. -*/ - rq->rl = rl; - } -#endif if (unlikely (!rq)) {/* * Allocation failed presumably due to memory. Undo anything
Re: [PATCH] io-controller: Add io group reference handling for request
Vivek Goyal wrote:
...
Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current
block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)
Hi Vivek,
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Quoted 20 lines
block/elevator-fq.c +6
-1{struct cgroup *cgroup;struct io_group *iog;struct elv_fq_data *efqd = &q->elevator->efqd;rcu_read_lock (); - cgroup =get_cgroup_from_bio (bio); + +if (curr) + cgroup =task_cgroup (current, io_subsys_id); +else + cgroup =get_cgroup_from_bio (bio); +if (!cgroup) {if (create) iog = efqd->root_group;@@ -1500,7 +1507,7 @@ out:rcu_read_unlock ();return iog; }
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 24 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)Hi Vivek,
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that get_cgroup_from_bio() is much more slow than task_cgroup() and need to lock/unlock_page_cgroup() in get_blkio_cgroup_id(), while task_cgroup() is rcu protected.
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can use task_cgroup().
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
-Andrea
Quoted 26 lines
Quoted 21 lines
block/elevator-fq.c +6
-1{struct cgroup *cgroup;struct io_group *iog;struct elv_fq_data *efqd = &q->elevator->efqd;rcu_read_lock (); - cgroup =get_cgroup_from_bio (bio); + +if (curr) + cgroup =task_cgroup (current, io_subsys_id); +else + cgroup =get_cgroup_from_bio (bio); +if (!cgroup) {if (create) iog = efqd->root_group;@@ -1500,7 +1507,7 @@ out:rcu_read_unlock ();return iog; }Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 31 lines · Andrea Righi
Quoted 23 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)Hi Vivek,
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.
Quoted 8 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Hi Andrea,
Thanks for pointing this out. Yes, i think we can determine io group in terms of bio->bi_rw. If bio is a READ bio, just taking io group by task_cgroup(). If it's a WRITE bio, getting it from blkio_cgroup.
Quoted 30 lines
-Andrea
Quoted 24 lines
Quoted 20 lines
block/elevator-fq.c +6
-1{struct cgroup *cgroup;struct io_group *iog;struct elv_fq_data *efqd = &q->elevator->efqd;rcu_read_lock (); - cgroup =get_cgroup_from_bio (bio); + +if (curr) + cgroup =task_cgroup (current, io_subsys_id); +else + cgroup =get_cgroup_from_bio (bio); +if (!cgroup) {if (create) iog = efqd->root_group;@@ -1500,7 +1507,7 @@ out:rcu_read_unlock ();return iog; }Signature · 2 lines
Regards Gui Jianfeng
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 41 lines · Gui Jianfeng
Quoted 31 lines · Andrea Righi
Quoted 23 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)Hi Vivek,
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.Quoted 8 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Quoted 6 lines
Hi Andrea,
Thanks for pointing this out. Yes, i think we can determine io group in terms of
bio->bi_rw. If bio is a READ bio, just taking io group bytask_cgroup(). If it's a WRITE bio, getting it fromblkio_cgroup.
Gui, we are already doing it. page tracking functionality is used only for async IO and for all sync IO, we are using submitting tasks's group to determine io group bio belongs to.
if (elv_bio_sync(bio)) {
/* sync io. Determine cgroup from submitting task
* context.*/
cgroup = task_cgroup(current, io_subsys_id);
return cgroup;
}Thanks
Vivek
Re: [PATCH] io-controller: Add io group reference handling for request
In reply to Andrea RighiQuoted 31 lines · Andrea Righi
Quoted 23 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)Hi Vivek,
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.
Quoted 8 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Hi Andrea,
Thanks for pointing this out. Yes, i think we can determine io group in terms of bio->bi_rw. If bio is a READ bio, just taking io group by task_cgroup(). If it's a WRITE bio, getting it from blkio_cgroup.
Quoted 30 lines
-Andrea
Quoted 24 lines
Quoted 20 lines
block/elevator-fq.c +6
-1{struct cgroup *cgroup;struct io_group *iog;struct elv_fq_data *efqd = &q->elevator->efqd;rcu_read_lock (); - cgroup =get_cgroup_from_bio (bio); + +if (curr) + cgroup =task_cgroup (current, io_subsys_id); +else + cgroup =get_cgroup_from_bio (bio); +if (!cgroup) {if (create) iog = efqd->root_group;@@ -1500,7 +1507,7 @@ out:rcu_read_unlock ();return iog; }Signature · 2 lines
Regards Gui Jianfeng
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] io-controller: Add io group reference handling for request
In reply to Andrea RighiQuoted 25 lines · Andrea Righi
Quoted 24 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.
True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().
Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use task_cgroup() for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look at elv_bio_sync(bio).
You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity? In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
Thanks
Vivek
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 56 lines · Vivek Goyal
Quoted 25 lines · Andrea Righi
Quoted 24 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).
Quoted 5 lines
You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity? In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in read_swap_cache_async():
@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
*/
__set_page_locked(new_page);
SetPageSwapBacked(new_page);
+ blkio_cgroup_set_owner(new_page, current->mm);
err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) {
/*
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the blkio_cgroup_set_owner().
I wonder if it would be better to attach the blkio_cgroup to the anonymous page only when swap-out occurs. I mean, just put the blkio_cgroup_set_owner() hook in try_to_umap() in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.
BTW, O_DIRECT is another case that is possible to optimize, because all the bios generated by direct IO occur in the same context of the current task.
-Andrea
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 74 lines · Andrea Righi
Quoted 56 lines · Vivek Goyal
Quoted 25 lines · Andrea Righi
Quoted 24 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).Quoted 5 lines
You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity? In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in
read_swap_cache_async():(quoted diff) +1
-0@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, */ __set_page_locked(new_page); SetPageSwapBacked(new_page); + blkio_cgroup_set_owner(new_page, current->mm); err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) { /*
Quoted 3 lines
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
But will this readahead be not initiated in the context of the task taking the page fault?
handle_pte_fault()
do_swap_page() swapin_readahead() read_swap_cache_async()
If yes, then swap reads issued will still be in the context of process and we should be fine?
Quoted 3 lines
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the
blkio_cgroup_set_owner().
Agreed.
Quoted 3 lines
I wonder if it would be better to attach the
blkio_cgroupto the anonymous page only when swap-out occurs.
Swap seems to be an interesting case in general. Somebody raised this question on lwn io controller article also. A user process never asked for swap activity. It is something enforced by kernel. So while doing some swap outs, it does not seem too fair to charge the write out to the process page belongs to and the fact of the matter may be that there is some other memory hungry application which is forcing these swap outs.
Keeping this in mind, should swap activity be considered as system activity and be charged to root group instead of to user tasks in other cgroups?
Quoted 8 lines
I mean, just put the
blkio_cgroup_set_owner()hook intry_to_umap()in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.BTW,
O_DIRECTis another case that is possible to optimize, because all the bios generated by direct IO occur in the same context of the current task.
Agreed about the direct IO optimization.
Ryo, what do you think? would you like to do include these optimizations by the Andrea in next version of IO tracking patches?
Thanks
Vivek
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 82 lines · Vivek Goyal
Quoted 74 lines · Andrea Righi
Quoted 56 lines · Vivek Goyal
Quoted 25 lines · Andrea Righi
Quoted 24 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).Quoted 5 lines
You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity? In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in
read_swap_cache_async():(quoted diff) +1
-0@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, */ __set_page_locked(new_page); SetPageSwapBacked(new_page); + blkio_cgroup_set_owner(new_page, current->mm); err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) { /*Quoted 3 lines
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
But will this readahead be not initiated in the context of the task taking the page fault?
Quoted 7 lines
handle_pte_fault()do_swap_page() swapin_readahead() read_swap_cache_async()If yes, then swap reads issued will still be in the context of process and we should be fine?
Right. I was trying to say that the current task may swap-in also pages belonging to a different task, so from a certain point of view it's not so fair to charge the current task for the whole activity. But ok, I think it's a minor issue.
Quoted 18 lines
Quoted 4 lines
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the
blkio_cgroup_set_owner().Agreed.
Quoted 3 lines
I wonder if it would be better to attach the
blkio_cgroupto the anonymous page only when swap-out occurs.Swap seems to be an interesting case in general. Somebody raised this question on lwn io controller article also. A user process never asked for swap activity. It is something enforced by kernel. So while doing some swap outs, it does not seem too fair to charge the write out to the process page belongs to and the fact of the matter may be that there is some other memory hungry application which is forcing these swap outs.
Quoted 3 lines
Keeping this in mind, should swap activity be considered as system activity and be charged to root group instead of to user tasks in other cgroups?
In this case I assume the swap-in activity should be charged to the root cgroup as well.
Anyway, in the logic of the memory and swap control it would seem reasonable to provide IO separation also for the swap IO activity.
In the MEMHOG example, it would be unfair if the memory pressure is caused by a task in another cgroup, but with memory and swap isolation a memory pressure condition can only be caused by a memory hog that runs in the same cgroup. From this point of view it seems more fair to consider the swap activity as the particular cgroup IO activity, instead of charging always the root cgroup.
Otherwise, I suspect, memory pressure would be a simple way to blow away any kind of QoS guarantees provided by the IO controller.
Quoted 10 lines
Quoted 9 lines
I mean, just put the
blkio_cgroup_set_owner()hook intry_to_umap()in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.BTW,
O_DIRECTis another case that is possible to optimize, because all the bios generated by direct IO occur in the same context of the current task.
Quoted 7 lines
Agreed about the direct IO optimization.
Ryo, what do you think? would you like to do include these optimizations by the Andrea in next version of IO tracking patches?
Thanks
Vivek
Thanks,
-Andrea
Re: [PATCH] io-controller: Add io group reference handling for request
Hi Andrea and Vivek,
From: Andrea Righi <[email protected]>
Subject: Re: [PATCH] io-controller: Add io group reference handling for request
Date: Mon, 18 May 2009 16:39:23 +0200
Quoted 54 lines · Vivek Goyal
Quoted 53 lines · Andrea Righi
Quoted 52 lines · Vivek Goyal
Quoted 51 lines · Andrea Righi
Quoted 25 lines · Gui Jianfeng
Quoted 24 lines
Vivek Goyal wrote:
...Quoted 9 lines
}
(quoted diff) +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru /* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. + * If "curr" is set, io group is information is searched for current + * task and not with the help of bio. + * + * FIXME: Can we assume that if bio is NULL then lookup group for currentReview struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
(quoted diff) +3
-5+ * task and not create extra function parameter ? * - * Note: There is a narrow window of race where a group is being freed - * by cgroup deletion path and some rq has slipped through in this group. - * Fix it. */ -struct io_group *io_get_io_group_bio(struct request_queue *q, struct bio *bio, - int create) +struct io_group *io_get_io_group(struct request_queue *q, struct bio *bio, + int create, int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity?
I think that there are some kernel threads (e.g., dm-crypt, LVM and md devices) which actually submit IOs instead of tasks which originate the IOs. When IOs are submitted from such kernel threads, we can't use submitting task's context to determine to which cgroup the IO belongs.
Quoted 70 lines
Quoted 22 lines
Quoted 15 lines
Quoted 3 lines
In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in
read_swap_cache_async():(quoted diff) +1
-0@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, */ __set_page_locked(new_page); SetPageSwapBacked(new_page); + blkio_cgroup_set_owner(new_page, current->mm); err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) { /*Quoted 3 lines
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
But will this readahead be not initiated in the context of the task taking the page fault?
Quoted 7 lines
handle_pte_fault()do_swap_page() swapin_readahead() read_swap_cache_async()If yes, then swap reads issued will still be in the context of process and we should be fine?
Right. I was trying to say that the current task may swap-in also pages belonging to a different task, so from a certain point of view it's not so fair to charge the current task for the whole activity. But ok, I think it's a minor issue.
Quoted 18 lines
Quoted 4 lines
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the
blkio_cgroup_set_owner().Agreed.
Quoted 3 lines
I wonder if it would be better to attach the
blkio_cgroupto the anonymous page only when swap-out occurs.Swap seems to be an interesting case in general. Somebody raised this question on lwn io controller article also. A user process never asked for swap activity. It is something enforced by kernel. So while doing some swap outs, it does not seem too fair to charge the write out to the process page belongs to and the fact of the matter may be that there is some other memory hungry application which is forcing these swap outs.
Quoted 3 lines
Keeping this in mind, should swap activity be considered as system activity and be charged to root group instead of to user tasks in other cgroups?
In this case I assume the swap-in activity should be charged to the root cgroup as well.
Anyway, in the logic of the memory and swap control it would seem reasonable to provide IO separation also for the swap IO activity.
In the MEMHOG example, it would be unfair if the memory pressure is caused by a task in another cgroup, but with memory and swap isolation a memory pressure condition can only be caused by a memory hog that runs in the same cgroup. From this point of view it seems more fair to consider the swap activity as the particular cgroup IO activity, instead of charging always the root cgroup.
Quoted 8 lines
Otherwise, I suspect, memory pressure would be a simple way to blow away any kind of QoS guarantees provided by the IO controller.
Quoted 5 lines
Quoted 5 lines
I mean, just put the
blkio_cgroup_set_owner()hook intry_to_umap()in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.
I think that only putting the hook in try_to_unmap() doesn't work correctly, because IOs will be charged to reclaiming processes or kswapd. These IOs should be charged to processes which cause memory pressure.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
BTW,
O_DIRECTis another case that is possible to optimize, because all the bios generated by direct IO occur in the same context of the current task.Agreed about the direct IO optimization.
Quoted 8 lines
Quoted 5 lines
Ryo, what do you think? would you like to do include these optimizations by the Andrea in next version of IO tracking patches?
Thanks
VivekThanks,
-Andrea
Thanks,
Ryo Tsuruta
Re: [PATCH] io-controller: Add io group reference handling for request
Hi Andrea and Vivek,
Quoted 153 lines · Ryo Tsuruta
Hi Andrea and Vivek,
From: Andrea Righi <[email protected]>
Subject: Re: [PATCH] io-controller: Add io group reference handling for request
Date: Mon, 18 May 2009 16:39:23 +0200Quoted 54 lines · Vivek Goyal
Quoted 53 lines · Andrea Righi
Quoted 52 lines · Vivek Goyal
Quoted 51 lines · Andrea Righi
Quoted 25 lines · Gui Jianfeng
Quoted 24 lines
Vivek Goyal wrote:
...Quoted 9 lines
}
(quoted diff) +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru /* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. + * If "curr" is set, io group is information is searched for current + * task and not with the help of bio. + * + * FIXME: Can we assume that if bio is NULL then lookup group for currentReview struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
(quoted diff) +3
-5+ * task and not create extra function parameter ? * - * Note: There is a narrow window of race where a group is being freed - * by cgroup deletion path and some rq has slipped through in this group. - * Fix it. */ -struct io_group *io_get_io_group_bio(struct request_queue *q, struct bio *bio, - int create) +struct io_group *io_get_io_group(struct request_queue *q, struct bio *bio, + int create, int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity?
I think that there are some kernel threads (e.g., dm-crypt, LVM and md devices) which actually submit IOs instead of tasks which originate the IOs. When IOs are submitted from such kernel threads, we can't use submitting task's context to determine to which cgroup the IO belongs.
Quoted 70 lines
Quoted 22 lines
Quoted 15 lines
Quoted 3 lines
In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in
read_swap_cache_async():(quoted diff) +1
-0@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, */ __set_page_locked(new_page); SetPageSwapBacked(new_page); + blkio_cgroup_set_owner(new_page, current->mm); err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) { /*Quoted 3 lines
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
But will this readahead be not initiated in the context of the task taking the page fault?
Quoted 7 lines
handle_pte_fault()do_swap_page() swapin_readahead() read_swap_cache_async()If yes, then swap reads issued will still be in the context of process and we should be fine?
Right. I was trying to say that the current task may swap-in also pages belonging to a different task, so from a certain point of view it's not so fair to charge the current task for the whole activity. But ok, I think it's a minor issue.
Quoted 18 lines
Quoted 4 lines
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the
blkio_cgroup_set_owner().Agreed.
Quoted 3 lines
I wonder if it would be better to attach the
blkio_cgroupto the anonymous page only when swap-out occurs.Swap seems to be an interesting case in general. Somebody raised this question on lwn io controller article also. A user process never asked for swap activity. It is something enforced by kernel. So while doing some swap outs, it does not seem too fair to charge the write out to the process page belongs to and the fact of the matter may be that there is some other memory hungry application which is forcing these swap outs.
Quoted 3 lines
Keeping this in mind, should swap activity be considered as system activity and be charged to root group instead of to user tasks in other cgroups?
In this case I assume the swap-in activity should be charged to the root cgroup as well.
Anyway, in the logic of the memory and swap control it would seem reasonable to provide IO separation also for the swap IO activity.
In the MEMHOG example, it would be unfair if the memory pressure is caused by a task in another cgroup, but with memory and swap isolation a memory pressure condition can only be caused by a memory hog that runs in the same cgroup. From this point of view it seems more fair to consider the swap activity as the particular cgroup IO activity, instead of charging always the root cgroup.
Quoted 8 lines
Otherwise, I suspect, memory pressure would be a simple way to blow away any kind of QoS guarantees provided by the IO controller.
Quoted 5 lines
Quoted 5 lines
I mean, just put the
blkio_cgroup_set_owner()hook intry_to_umap()in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.
Quoted 4 lines
I think that only putting the hook in
try_to_unmap()doesn't work correctly, because IOs will be charged to reclaiming processes or kswapd. These IOs should be charged to processes which cause memory pressure.
Consider the following case:
(1) There are two processes Proc-A and Proc-B.
(2) Proc-A maps a large file into many pages by mmap() and writes many data to the file.
(3) After (2), Proc-B try to get a page, but there are no available pages because Proc-A has used them.
(4) kernel starts to reclaim pages, call try_to_unmap() to unmap a page which is owned by Proc-A, then blkio_cgroup_set_owner() sets Proc-B's ID on the page because the task's context is Proc-B.
(5) After (4), kernel writes the page out to a disk. This IO is charged to Proc-B.
In the above case, I think that the IO should be charged to a Proc-A, because the IO is caused by Proc-A's memory pressure.
I think we should consider in the case without memory and swap isolation.
Thanks,
Ryo Tsuruta
Quoted 17 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
BTW,
O_DIRECTis another case that is possible to optimize, because all the bios generated by direct IO occur in the same context of the current task.Agreed about the direct IO optimization.
Quoted 8 lines
Quoted 5 lines
Ryo, what do you think? would you like to do include these optimizations by the Andrea in next version of IO tracking patches?
Thanks
VivekThanks,
-AndreaThanks,
Ryo Tsuruta
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 18 lines · Ryo Tsuruta
Quoted 4 lines
I think that only putting the hook in
try_to_unmap()doesn't work correctly, because IOs will be charged to reclaiming processes or kswapd. These IOs should be charged to processes which cause memory pressure.Consider the following case:
(1) There are two processes Proc-A and Proc-B.
(2) Proc-A maps a large file into many pages bymmap()and writes many data to the file.
(3) After (2), Proc-B try to get a page, but there are no available pages because Proc-A has used them.
(4) kernel starts to reclaim pages, calltry_to_unmap()to unmap a page which is owned by Proc-A, thenblkio_cgroup_set_owner()sets Proc-B's ID on the page because the task's context is Proc-B.
(5) After (4), kernel writes the page out to a disk. This IO is charged to Proc-B.
Quoted 4 lines
In the above case, I think that the IO should be charged to a Proc-A, because the IO is caused by Proc-A's memory pressure.
I think we should consider in the case without memory and swap isolation.
mmmh.. even if they're strictly related I think we're mixing two different problems in this way: memory pressure control and IO control.
It seems you're proposing something like the badness() for OOM conditions to charge swap IO depending on how bad is a cgroup in terms of memory consumption. I don't think this is the right way to proceed, also because we already have the memory and swap control.
-Andrea
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 24 lines · Andrea Righi
Quoted 18 lines · Ryo Tsuruta
Quoted 4 lines
I think that only putting the hook in
try_to_unmap()doesn't work correctly, because IOs will be charged to reclaiming processes or kswapd. These IOs should be charged to processes which cause memory pressure.Consider the following case:
(1) There are two processes Proc-A and Proc-B.
(2) Proc-A maps a large file into many pages bymmap()and writes many data to the file.
(3) After (2), Proc-B try to get a page, but there are no available pages because Proc-A has used them.
(4) kernel starts to reclaim pages, calltry_to_unmap()to unmap a page which is owned by Proc-A, thenblkio_cgroup_set_owner()sets Proc-B's ID on the page because the task's context is Proc-B.
(5) After (4), kernel writes the page out to a disk. This IO is charged to Proc-B.Quoted 4 lines
In the above case, I think that the IO should be charged to a Proc-A, because the IO is caused by Proc-A's memory pressure.
I think we should consider in the case without memory and swap isolation.
Quoted 7 lines
mmmh.. even if they're strictly related I think we're mixing two different problems in this way: memory pressure control and IO control.
It seems you're proposing something like the
badness()for OOM conditions to charge swap IO depending on how bad is a cgroup in terms of memory consumption. I don't think this is the right way to proceed, also because we already have the memory and swap control.
cgroups support multiple hierarchy and it allows to have different divisions of tasks among hierarchy like below:
PIDs
mem+swap /hier1/grp1/tasks <= 1, 2, 3, 4
blkio /hier2/grp2/tasks <= 1, 2
grp3/tasks <= 3, 4
Don't we need to consider this case?
Thanks,
Ryo Tsuruta
Re: [PATCH] io-controller: Add io group reference handling for request
In reply to Andrea RighiQuoted 24 lines · Andrea Righi
Quoted 18 lines · Ryo Tsuruta
Quoted 4 lines
I think that only putting the hook in
try_to_unmap()doesn't work correctly, because IOs will be charged to reclaiming processes or kswapd. These IOs should be charged to processes which cause memory pressure.Consider the following case:
(1) There are two processes Proc-A and Proc-B.
(2) Proc-A maps a large file into many pages bymmap()and writes many data to the file.
(3) After (2), Proc-B try to get a page, but there are no available pages because Proc-A has used them.
(4) kernel starts to reclaim pages, calltry_to_unmap()to unmap a page which is owned by Proc-A, thenblkio_cgroup_set_owner()sets Proc-B's ID on the page because the task's context is Proc-B.
(5) After (4), kernel writes the page out to a disk. This IO is charged to Proc-B.Quoted 4 lines
In the above case, I think that the IO should be charged to a Proc-A, because the IO is caused by Proc-A's memory pressure.
I think we should consider in the case without memory and swap isolation.
Quoted 7 lines
mmmh.. even if they're strictly related I think we're mixing two different problems in this way: memory pressure control and IO control.
It seems you're proposing something like the
badness()for OOM conditions to charge swap IO depending on how bad is a cgroup in terms of memory consumption. I don't think this is the right way to proceed, also because we already have the memory and swap control.
cgroups support multiple hierarchy and it allows to have different divisions of tasks among hierarchy like below:
PIDs
mem+swap /hier1/grp1/tasks <= 1, 2, 3, 4
blkio /hier2/grp2/tasks <= 1, 2
grp3/tasks <= 3, 4
Don't we need to consider this case?
Thanks,
Ryo Tsuruta
A message that is not in the archive (sent off-list or not yet received): 20090527.155631.226800550.ryov-jCdQPDEk3idL9jVzuh4AOg@public.gmane.org
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 18 lines · Ryo Tsuruta
Quoted 4 lines
I think that only putting the hook in
try_to_unmap()doesn't work correctly, because IOs will be charged to reclaiming processes or kswapd. These IOs should be charged to processes which cause memory pressure.Consider the following case:
(1) There are two processes Proc-A and Proc-B.
(2) Proc-A maps a large file into many pages bymmap()and writes many data to the file.
(3) After (2), Proc-B try to get a page, but there are no available pages because Proc-A has used them.
(4) kernel starts to reclaim pages, calltry_to_unmap()to unmap a page which is owned by Proc-A, thenblkio_cgroup_set_owner()sets Proc-B's ID on the page because the task's context is Proc-B.
(5) After (4), kernel writes the page out to a disk. This IO is charged to Proc-B.
Quoted 4 lines
In the above case, I think that the IO should be charged to a Proc-A, because the IO is caused by Proc-A's memory pressure.
I think we should consider in the case without memory and swap isolation.
mmmh.. even if they're strictly related I think we're mixing two different problems in this way: memory pressure control and IO control.
It seems you're proposing something like the badness() for OOM conditions to charge swap IO depending on how bad is a cgroup in terms of memory consumption. I don't think this is the right way to proceed, also because we already have the memory and swap control.
-Andrea
Re: [PATCH] io-controller: Add io group reference handling for request
In reply to a message not in the archiveQuoted 174 lines · Ryo Tsuruta
Hi Andrea and Vivek,
Quoted 153 lines · Ryo Tsuruta
Hi Andrea and Vivek,
From: Andrea Righi <[email protected]>
Subject: Re: [PATCH] io-controller: Add io group reference handling for request
Date: Mon, 18 May 2009 16:39:23 +0200Quoted 54 lines · Vivek Goyal
Quoted 53 lines · Andrea Righi
Quoted 52 lines · Vivek Goyal
Quoted 51 lines · Andrea Righi
Quoted 25 lines · Gui Jianfeng
Quoted 24 lines
Vivek Goyal wrote:
...Quoted 9 lines
}
(quoted diff) +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru /* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. + * If "curr" is set, io group is information is searched for current + * task and not with the help of bio. + * + * FIXME: Can we assume that if bio is NULL then lookup group for currentReview struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
(quoted diff) +3
-5+ * task and not create extra function parameter ? * - * Note: There is a narrow window of race where a group is being freed - * by cgroup deletion path and some rq has slipped through in this group. - * Fix it. */ -struct io_group *io_get_io_group_bio(struct request_queue *q, struct bio *bio, - int create) +struct io_group *io_get_io_group(struct request_queue *q, struct bio *bio, + int create, int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity?
I think that there are some kernel threads (e.g., dm-crypt, LVM and md devices) which actually submit IOs instead of tasks which originate the IOs. When IOs are submitted from such kernel threads, we can't use submitting task's context to determine to which cgroup the IO belongs.
Quoted 70 lines
Quoted 22 lines
Quoted 15 lines
Quoted 3 lines
In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in
read_swap_cache_async():(quoted diff) +1
-0@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, */ __set_page_locked(new_page); SetPageSwapBacked(new_page); + blkio_cgroup_set_owner(new_page, current->mm); err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) { /*Quoted 3 lines
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
But will this readahead be not initiated in the context of the task taking the page fault?
Quoted 7 lines
handle_pte_fault()do_swap_page() swapin_readahead() read_swap_cache_async()If yes, then swap reads issued will still be in the context of process and we should be fine?
Right. I was trying to say that the current task may swap-in also pages belonging to a different task, so from a certain point of view it's not so fair to charge the current task for the whole activity. But ok, I think it's a minor issue.
Quoted 18 lines
Quoted 4 lines
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the
blkio_cgroup_set_owner().Agreed.
Quoted 3 lines
I wonder if it would be better to attach the
blkio_cgroupto the anonymous page only when swap-out occurs.Swap seems to be an interesting case in general. Somebody raised this question on lwn io controller article also. A user process never asked for swap activity. It is something enforced by kernel. So while doing some swap outs, it does not seem too fair to charge the write out to the process page belongs to and the fact of the matter may be that there is some other memory hungry application which is forcing these swap outs.
Quoted 3 lines
Keeping this in mind, should swap activity be considered as system activity and be charged to root group instead of to user tasks in other cgroups?
In this case I assume the swap-in activity should be charged to the root cgroup as well.
Anyway, in the logic of the memory and swap control it would seem reasonable to provide IO separation also for the swap IO activity.
In the MEMHOG example, it would be unfair if the memory pressure is caused by a task in another cgroup, but with memory and swap isolation a memory pressure condition can only be caused by a memory hog that runs in the same cgroup. From this point of view it seems more fair to consider the swap activity as the particular cgroup IO activity, instead of charging always the root cgroup.
Quoted 8 lines
Otherwise, I suspect, memory pressure would be a simple way to blow away any kind of QoS guarantees provided by the IO controller.
Quoted 5 lines
Quoted 5 lines
I mean, just put the
blkio_cgroup_set_owner()hook intry_to_umap()in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.Quoted 4 lines
I think that only putting the hook in
try_to_unmap()doesn't work correctly, because IOs will be charged to reclaiming processes or kswapd. These IOs should be charged to processes which cause memory pressure.Consider the following case:
(1) There are two processes Proc-A and Proc-B.
(2) Proc-A maps a large file into many pages bymmap()and writes many data to the file.
(3) After (2), Proc-B try to get a page, but there are no available pages because Proc-A has used them.
(4) kernel starts to reclaim pages, calltry_to_unmap()to unmap a page which is owned by Proc-A, thenblkio_cgroup_set_owner()sets Proc-B's ID on the page because the task's context is Proc-B.
(5) After (4), kernel writes the page out to a disk. This IO is charged to Proc-B.
Quoted 5 lines
In the above case, I think that the IO should be charged to a Proc-A, because the IO is caused by Proc-A's memory pressure.
I think we should consider in the case without memory and swap isolation.
But what happens if Proc-B is consuming lots of memory and then process A asks for one page of memory and that triggers the memory reclaim. In that case we are kind of penalizing process A from IO point of view because some other process consumed lots of memory?
So it looks like that if one mounts mem+swap and io controller on same hierarchy, then things probably would be fine as swap IO generated due to either memory pressure or periodic reclaim by kswapd, will be charged to right cgroup.
But if they are not mounted on same hiearchy, then I guess it is not too bad to charge the owner of the page for swap IO. It is not very accurate but at the same time there does not seem to be an easy way out?
Thanks
Vivek
Re: [PATCH] io-controller: Add io group reference handling for request
In reply to Ryo TsurutaQuoted 174 lines · Ryo Tsuruta
Hi Andrea and Vivek,
Quoted 153 lines · Ryo Tsuruta
Hi Andrea and Vivek,
From: Andrea Righi <[email protected]>
Subject: Re: [PATCH] io-controller: Add io group reference handling for request
Date: Mon, 18 May 2009 16:39:23 +0200Quoted 54 lines · Vivek Goyal
Quoted 53 lines · Andrea Righi
Quoted 52 lines · Vivek Goyal
Quoted 51 lines · Andrea Righi
Quoted 25 lines · Gui Jianfeng
Quoted 24 lines
Vivek Goyal wrote:
...Quoted 9 lines
}
(quoted diff) +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru /* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. + * If "curr" is set, io group is information is searched for current + * task and not with the help of bio. + * + * FIXME: Can we assume that if bio is NULL then lookup group for currentReview struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
(quoted diff) +3
-5+ * task and not create extra function parameter ? * - * Note: There is a narrow window of race where a group is being freed - * by cgroup deletion path and some rq has slipped through in this group. - * Fix it. */ -struct io_group *io_get_io_group_bio(struct request_queue *q, struct bio *bio, - int create) +struct io_group *io_get_io_group(struct request_queue *q, struct bio *bio, + int create, int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity?
I think that there are some kernel threads (e.g., dm-crypt, LVM and md devices) which actually submit IOs instead of tasks which originate the IOs. When IOs are submitted from such kernel threads, we can't use submitting task's context to determine to which cgroup the IO belongs.
Quoted 70 lines
Quoted 22 lines
Quoted 15 lines
Quoted 3 lines
In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in
read_swap_cache_async():(quoted diff) +1
-0@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, */ __set_page_locked(new_page); SetPageSwapBacked(new_page); + blkio_cgroup_set_owner(new_page, current->mm); err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) { /*Quoted 3 lines
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
But will this readahead be not initiated in the context of the task taking the page fault?
Quoted 7 lines
handle_pte_fault()do_swap_page() swapin_readahead() read_swap_cache_async()If yes, then swap reads issued will still be in the context of process and we should be fine?
Right. I was trying to say that the current task may swap-in also pages belonging to a different task, so from a certain point of view it's not so fair to charge the current task for the whole activity. But ok, I think it's a minor issue.
Quoted 18 lines
Quoted 4 lines
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the
blkio_cgroup_set_owner().Agreed.
Quoted 3 lines
I wonder if it would be better to attach the
blkio_cgroupto the anonymous page only when swap-out occurs.Swap seems to be an interesting case in general. Somebody raised this question on lwn io controller article also. A user process never asked for swap activity. It is something enforced by kernel. So while doing some swap outs, it does not seem too fair to charge the write out to the process page belongs to and the fact of the matter may be that there is some other memory hungry application which is forcing these swap outs.
Quoted 3 lines
Keeping this in mind, should swap activity be considered as system activity and be charged to root group instead of to user tasks in other cgroups?
In this case I assume the swap-in activity should be charged to the root cgroup as well.
Anyway, in the logic of the memory and swap control it would seem reasonable to provide IO separation also for the swap IO activity.
In the MEMHOG example, it would be unfair if the memory pressure is caused by a task in another cgroup, but with memory and swap isolation a memory pressure condition can only be caused by a memory hog that runs in the same cgroup. From this point of view it seems more fair to consider the swap activity as the particular cgroup IO activity, instead of charging always the root cgroup.
Quoted 8 lines
Otherwise, I suspect, memory pressure would be a simple way to blow away any kind of QoS guarantees provided by the IO controller.
Quoted 5 lines
Quoted 5 lines
I mean, just put the
blkio_cgroup_set_owner()hook intry_to_umap()in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.Quoted 4 lines
I think that only putting the hook in
try_to_unmap()doesn't work correctly, because IOs will be charged to reclaiming processes or kswapd. These IOs should be charged to processes which cause memory pressure.Consider the following case:
(1) There are two processes Proc-A and Proc-B.
(2) Proc-A maps a large file into many pages bymmap()and writes many data to the file.
(3) After (2), Proc-B try to get a page, but there are no available pages because Proc-A has used them.
(4) kernel starts to reclaim pages, calltry_to_unmap()to unmap a page which is owned by Proc-A, thenblkio_cgroup_set_owner()sets Proc-B's ID on the page because the task's context is Proc-B.
(5) After (4), kernel writes the page out to a disk. This IO is charged to Proc-B.
Quoted 5 lines
In the above case, I think that the IO should be charged to a Proc-A, because the IO is caused by Proc-A's memory pressure.
I think we should consider in the case without memory and swap isolation.
But what happens if Proc-B is consuming lots of memory and then process A asks for one page of memory and that triggers the memory reclaim. In that case we are kind of penalizing process A from IO point of view because some other process consumed lots of memory?
So it looks like that if one mounts mem+swap and io controller on same hierarchy, then things probably would be fine as swap IO generated due to either memory pressure or periodic reclaim by kswapd, will be charged to right cgroup.
But if they are not mounted on same hiearchy, then I guess it is not too bad to charge the owner of the page for swap IO. It is not very accurate but at the same time there does not seem to be an easy way out?
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): 20090526.203424.39179999.ryov-jCdQPDEk3idL9jVzuh4AOg@public.gmane.org
Re: [PATCH] io-controller: Add io group reference handling for request
Hi Andrea and Vivek,
Quoted 153 lines · Ryo Tsuruta
Hi Andrea and Vivek,
From: Andrea Righi <[email protected]>
Subject: Re: [PATCH] io-controller: Add io group reference handling for request
Date: Mon, 18 May 2009 16:39:23 +0200Quoted 54 lines · Vivek Goyal
Quoted 53 lines · Andrea Righi
Quoted 52 lines · Vivek Goyal
Quoted 51 lines · Andrea Righi
Quoted 25 lines · Gui Jianfeng
Quoted 24 lines
Vivek Goyal wrote:
...Quoted 9 lines
}
(quoted diff) +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru /* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. + * If "curr" is set, io group is information is searched for current + * task and not with the help of bio. + * + * FIXME: Can we assume that if bio is NULL then lookup group for currentReview struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
(quoted diff) +3
-5+ * task and not create extra function parameter ? * - * Note: There is a narrow window of race where a group is being freed - * by cgroup deletion path and some rq has slipped through in this group. - * Fix it. */ -struct io_group *io_get_io_group_bio(struct request_queue *q, struct bio *bio, - int create) +struct io_group *io_get_io_group(struct request_queue *q, struct bio *bio, + int create, int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity?
I think that there are some kernel threads (e.g., dm-crypt, LVM and md devices) which actually submit IOs instead of tasks which originate the IOs. When IOs are submitted from such kernel threads, we can't use submitting task's context to determine to which cgroup the IO belongs.
Quoted 70 lines
Quoted 22 lines
Quoted 15 lines
Quoted 3 lines
In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in
read_swap_cache_async():(quoted diff) +1
-0@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, */ __set_page_locked(new_page); SetPageSwapBacked(new_page); + blkio_cgroup_set_owner(new_page, current->mm); err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) { /*Quoted 3 lines
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
But will this readahead be not initiated in the context of the task taking the page fault?
Quoted 7 lines
handle_pte_fault()do_swap_page() swapin_readahead() read_swap_cache_async()If yes, then swap reads issued will still be in the context of process and we should be fine?
Right. I was trying to say that the current task may swap-in also pages belonging to a different task, so from a certain point of view it's not so fair to charge the current task for the whole activity. But ok, I think it's a minor issue.
Quoted 18 lines
Quoted 4 lines
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the
blkio_cgroup_set_owner().Agreed.
Quoted 3 lines
I wonder if it would be better to attach the
blkio_cgroupto the anonymous page only when swap-out occurs.Swap seems to be an interesting case in general. Somebody raised this question on lwn io controller article also. A user process never asked for swap activity. It is something enforced by kernel. So while doing some swap outs, it does not seem too fair to charge the write out to the process page belongs to and the fact of the matter may be that there is some other memory hungry application which is forcing these swap outs.
Quoted 3 lines
Keeping this in mind, should swap activity be considered as system activity and be charged to root group instead of to user tasks in other cgroups?
In this case I assume the swap-in activity should be charged to the root cgroup as well.
Anyway, in the logic of the memory and swap control it would seem reasonable to provide IO separation also for the swap IO activity.
In the MEMHOG example, it would be unfair if the memory pressure is caused by a task in another cgroup, but with memory and swap isolation a memory pressure condition can only be caused by a memory hog that runs in the same cgroup. From this point of view it seems more fair to consider the swap activity as the particular cgroup IO activity, instead of charging always the root cgroup.
Quoted 8 lines
Otherwise, I suspect, memory pressure would be a simple way to blow away any kind of QoS guarantees provided by the IO controller.
Quoted 5 lines
Quoted 5 lines
I mean, just put the
blkio_cgroup_set_owner()hook intry_to_umap()in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.
Quoted 4 lines
I think that only putting the hook in
try_to_unmap()doesn't work correctly, because IOs will be charged to reclaiming processes or kswapd. These IOs should be charged to processes which cause memory pressure.
Consider the following case:
(1) There are two processes Proc-A and Proc-B.
(2) Proc-A maps a large file into many pages by mmap() and writes many data to the file.
(3) After (2), Proc-B try to get a page, but there are no available pages because Proc-A has used them.
(4) kernel starts to reclaim pages, call try_to_unmap() to unmap a page which is owned by Proc-A, then blkio_cgroup_set_owner() sets Proc-B's ID on the page because the task's context is Proc-B.
(5) After (4), kernel writes the page out to a disk. This IO is charged to Proc-B.
In the above case, I think that the IO should be charged to a Proc-A, because the IO is caused by Proc-A's memory pressure.
I think we should consider in the case without memory and swap isolation.
Thanks,
Ryo Tsuruta
Quoted 17 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
BTW,
O_DIRECTis another case that is possible to optimize, because all the bios generated by direct IO occur in the same context of the current task.Agreed about the direct IO optimization.
Quoted 8 lines
Quoted 5 lines
Ryo, what do you think? would you like to do include these optimizations by the Andrea in next version of IO tracking patches?
Thanks
VivekThanks,
-AndreaThanks,
Ryo Tsuruta
Re: [PATCH] io-controller: Add io group reference handling for request
In reply to Andrea RighiHi Andrea and Vivek,
From: Andrea Righi <[email protected]>
Subject: Re: [PATCH] io-controller: Add io group reference handling for request
Date: Mon, 18 May 2009 16:39:23 +0200
Quoted 54 lines · Vivek Goyal
Quoted 53 lines · Andrea Righi
Quoted 52 lines · Vivek Goyal
Quoted 51 lines · Andrea Righi
Quoted 25 lines · Gui Jianfeng
Quoted 24 lines
Vivek Goyal wrote:
...Quoted 9 lines
}
(quoted diff) +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru /* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. + * If "curr" is set, io group is information is searched for current + * task and not with the help of bio. + * + * FIXME: Can we assume that if bio is NULL then lookup group for currentReview struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
(quoted diff) +3
-5+ * task and not create extra function parameter ? * - * Note: There is a narrow window of race where a group is being freed - * by cgroup deletion path and some rq has slipped through in this group. - * Fix it. */ -struct io_group *io_get_io_group_bio(struct request_queue *q, struct bio *bio, - int create) +struct io_group *io_get_io_group(struct request_queue *q, struct bio *bio, + int create, int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity?
I think that there are some kernel threads (e.g., dm-crypt, LVM and md devices) which actually submit IOs instead of tasks which originate the IOs. When IOs are submitted from such kernel threads, we can't use submitting task's context to determine to which cgroup the IO belongs.
Quoted 70 lines
Quoted 22 lines
Quoted 15 lines
Quoted 3 lines
In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in
read_swap_cache_async():(quoted diff) +1
-0@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, */ __set_page_locked(new_page); SetPageSwapBacked(new_page); + blkio_cgroup_set_owner(new_page, current->mm); err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) { /*Quoted 3 lines
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
But will this readahead be not initiated in the context of the task taking the page fault?
Quoted 7 lines
handle_pte_fault()do_swap_page() swapin_readahead() read_swap_cache_async()If yes, then swap reads issued will still be in the context of process and we should be fine?
Right. I was trying to say that the current task may swap-in also pages belonging to a different task, so from a certain point of view it's not so fair to charge the current task for the whole activity. But ok, I think it's a minor issue.
Quoted 18 lines
Quoted 4 lines
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the
blkio_cgroup_set_owner().Agreed.
Quoted 3 lines
I wonder if it would be better to attach the
blkio_cgroupto the anonymous page only when swap-out occurs.Swap seems to be an interesting case in general. Somebody raised this question on lwn io controller article also. A user process never asked for swap activity. It is something enforced by kernel. So while doing some swap outs, it does not seem too fair to charge the write out to the process page belongs to and the fact of the matter may be that there is some other memory hungry application which is forcing these swap outs.
Quoted 3 lines
Keeping this in mind, should swap activity be considered as system activity and be charged to root group instead of to user tasks in other cgroups?
In this case I assume the swap-in activity should be charged to the root cgroup as well.
Anyway, in the logic of the memory and swap control it would seem reasonable to provide IO separation also for the swap IO activity.
In the MEMHOG example, it would be unfair if the memory pressure is caused by a task in another cgroup, but with memory and swap isolation a memory pressure condition can only be caused by a memory hog that runs in the same cgroup. From this point of view it seems more fair to consider the swap activity as the particular cgroup IO activity, instead of charging always the root cgroup.
Quoted 8 lines
Otherwise, I suspect, memory pressure would be a simple way to blow away any kind of QoS guarantees provided by the IO controller.
Quoted 5 lines
Quoted 5 lines
I mean, just put the
blkio_cgroup_set_owner()hook intry_to_umap()in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.
I think that only putting the hook in try_to_unmap() doesn't work correctly, because IOs will be charged to reclaiming processes or kswapd. These IOs should be charged to processes which cause memory pressure.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
BTW,
O_DIRECTis another case that is possible to optimize, because all the bios generated by direct IO occur in the same context of the current task.Agreed about the direct IO optimization.
Quoted 8 lines
Quoted 5 lines
Ryo, what do you think? would you like to do include these optimizations by the Andrea in next version of IO tracking patches?
Thanks
VivekThanks,
-Andrea
Thanks,
Ryo Tsuruta
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 82 lines · Vivek Goyal
Quoted 74 lines · Andrea Righi
Quoted 56 lines · Vivek Goyal
Quoted 25 lines · Andrea Righi
Quoted 24 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 9 lines
}
(quoted diff) +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru /* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. + * If "curr" is set, io group is information is searched for current + * task and not with the help of bio. + * + * FIXME: Can we assume that if bio is NULL then lookup group for currentReview struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
(quoted diff) +3
-5+ * task and not create extra function parameter ? * - * Note: There is a narrow window of race where a group is being freed - * by cgroup deletion path and some rq has slipped through in this group. - * Fix it. */ -struct io_group *io_get_io_group_bio(struct request_queue *q, struct bio *bio, - int create) +struct io_group *io_get_io_group(struct request_queue *q, struct bio *bio, + int create, int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).Quoted 5 lines
You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity? In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in
read_swap_cache_async():(quoted diff) +1
-0@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, */ __set_page_locked(new_page); SetPageSwapBacked(new_page); + blkio_cgroup_set_owner(new_page, current->mm); err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) { /*Quoted 3 lines
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
But will this readahead be not initiated in the context of the task taking the page fault?
Quoted 7 lines
handle_pte_fault()do_swap_page() swapin_readahead() read_swap_cache_async()If yes, then swap reads issued will still be in the context of process and we should be fine?
Right. I was trying to say that the current task may swap-in also pages belonging to a different task, so from a certain point of view it's not so fair to charge the current task for the whole activity. But ok, I think it's a minor issue.
Quoted 18 lines
Quoted 4 lines
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the
blkio_cgroup_set_owner().Agreed.
Quoted 3 lines
I wonder if it would be better to attach the
blkio_cgroupto the anonymous page only when swap-out occurs.Swap seems to be an interesting case in general. Somebody raised this question on lwn io controller article also. A user process never asked for swap activity. It is something enforced by kernel. So while doing some swap outs, it does not seem too fair to charge the write out to the process page belongs to and the fact of the matter may be that there is some other memory hungry application which is forcing these swap outs.
Quoted 3 lines
Keeping this in mind, should swap activity be considered as system activity and be charged to root group instead of to user tasks in other cgroups?
In this case I assume the swap-in activity should be charged to the root cgroup as well.
Anyway, in the logic of the memory and swap control it would seem reasonable to provide IO separation also for the swap IO activity.
In the MEMHOG example, it would be unfair if the memory pressure is caused by a task in another cgroup, but with memory and swap isolation a memory pressure condition can only be caused by a memory hog that runs in the same cgroup. From this point of view it seems more fair to consider the swap activity as the particular cgroup IO activity, instead of charging always the root cgroup.
Otherwise, I suspect, memory pressure would be a simple way to blow away any kind of QoS guarantees provided by the IO controller.
Quoted 10 lines
Quoted 9 lines
I mean, just put the
blkio_cgroup_set_owner()hook intry_to_umap()in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.BTW,
O_DIRECTis another case that is possible to optimize, because all the bios generated by direct IO occur in the same context of the current task.
Quoted 7 lines
Agreed about the direct IO optimization.
Ryo, what do you think? would you like to do include these optimizations by the Andrea in next version of IO tracking patches?
Thanks
Vivek
Thanks,
-Andrea
Re: [PATCH] io-controller: Add io group reference handling for request
In reply to a message not in the archiveFrom: Vivek Goyal <[email protected]>
Subject: Re: [PATCH] io-controller: Add io group reference handling for request
Date: Mon, 18 May 2009 10:01:14 -0400
Quoted 116 lines · Andrea Righi
Quoted 74 lines · Vivek Goyal
Quoted 56 lines · Andrea Righi
Quoted 25 lines · Gui Jianfeng
Quoted 24 lines
Vivek Goyal wrote:
...Quoted 9 lines
}
(quoted diff) +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru /* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. + * If "curr" is set, io group is information is searched for current + * task and not with the help of bio. + * + * FIXME: Can we assume that if bio is NULL then lookup group for currentReview struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
(quoted diff) +3
-5+ * task and not create extra function parameter ? * - * Note: There is a narrow window of race where a group is being freed - * by cgroup deletion path and some rq has slipped through in this group. - * Fix it. */ -struct io_group *io_get_io_group_bio(struct request_queue *q, struct bio *bio, - int create) +struct io_group *io_get_io_group(struct request_queue *q, struct bio *bio, + int create, int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).Quoted 5 lines
You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity? In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in
read_swap_cache_async():(quoted diff) +1
-0@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, */ __set_page_locked(new_page); SetPageSwapBacked(new_page); + blkio_cgroup_set_owner(new_page, current->mm); err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) { /*Quoted 3 lines
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
But will this readahead be not initiated in the context of the task taking the page fault?
handle_pte_fault()do_swap_page() swapin_readahead() read_swap_cache_async()If yes, then swap reads issued will still be in the context of process and we should be fine?
Quoted 3 lines
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the
blkio_cgroup_set_owner().Agreed.
Quoted 3 lines
I wonder if it would be better to attach the
blkio_cgroupto the anonymous page only when swap-out occurs.Swap seems to be an interesting case in general. Somebody raised this question on lwn io controller article also. A user process never asked for swap activity. It is something enforced by kernel. So while doing some swap outs, it does not seem too fair to charge the write out to the process page belongs to and the fact of the matter may be that there is some other memory hungry application which is forcing these swap outs.
Keeping this in mind, should swap activity be considered as system activity and be charged to root group instead of to user tasks in other cgroups?
Quoted 5 lines
I mean, just put the
blkio_cgroup_set_owner()hook intry_to_umap()in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.
Quoted 8 lines
Quoted 3 lines
BTW,
O_DIRECTis another case that is possible to optimize, because all the bios generated by direct IO occur in the same context of the current task.Agreed about the direct IO optimization.
Ryo, what do you think? would you like to do include these optimizations by the Andrea in next version of IO tracking patches?
I'll consider whether these optimizations are reasonable.
Thanks,
Ryo Tsuruta
Re: [PATCH] io-controller: Add io group reference handling for request
In reply to Vivek GoyalFrom: Vivek Goyal <[email protected]>
Subject: Re: [PATCH] io-controller: Add io group reference handling for request
Date: Mon, 18 May 2009 10:01:14 -0400
Quoted 116 lines · Andrea Righi
Quoted 74 lines · Vivek Goyal
Quoted 56 lines · Andrea Righi
Quoted 25 lines · Gui Jianfeng
Quoted 24 lines
Vivek Goyal wrote:
...Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).Quoted 5 lines
You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity? In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in
read_swap_cache_async():(quoted diff) +1
-0@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, */ __set_page_locked(new_page); SetPageSwapBacked(new_page); + blkio_cgroup_set_owner(new_page, current->mm); err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) { /*Quoted 3 lines
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
But will this readahead be not initiated in the context of the task taking the page fault?
handle_pte_fault()do_swap_page() swapin_readahead() read_swap_cache_async()If yes, then swap reads issued will still be in the context of process and we should be fine?
Quoted 3 lines
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the
blkio_cgroup_set_owner().Agreed.
Quoted 3 lines
I wonder if it would be better to attach the
blkio_cgroupto the anonymous page only when swap-out occurs.Swap seems to be an interesting case in general. Somebody raised this question on lwn io controller article also. A user process never asked for swap activity. It is something enforced by kernel. So while doing some swap outs, it does not seem too fair to charge the write out to the process page belongs to and the fact of the matter may be that there is some other memory hungry application which is forcing these swap outs.
Keeping this in mind, should swap activity be considered as system activity and be charged to root group instead of to user tasks in other cgroups?
Quoted 5 lines
I mean, just put the
blkio_cgroup_set_owner()hook intry_to_umap()in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.
Quoted 8 lines
Quoted 3 lines
BTW,
O_DIRECTis another case that is possible to optimize, because all the bios generated by direct IO occur in the same context of the current task.Agreed about the direct IO optimization.
Ryo, what do you think? would you like to do include these optimizations by the Andrea in next version of IO tracking patches?
I'll consider whether these optimizations are reasonable.
Thanks,
Ryo Tsuruta
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 56 lines · Vivek Goyal
Quoted 25 lines · Andrea Righi
Quoted 24 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use
task_cgroup()for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look atelv_bio_sync(bio).
Quoted 5 lines
You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity? In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
No, for example in read_swap_cache_async():
@@ -308,6 +309,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
*/
__set_page_locked(new_page);
SetPageSwapBacked(new_page);
+ blkio_cgroup_set_owner(new_page, current->mm);
err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); if (likely(!err)) {
/*
This is a read, but the current task is not always the owner of this swap cache page, because it's a readahead operation.
Anyway, this is a minor corner case I think. And probably it is safe to consider this like any other read IO and get rid of the blkio_cgroup_set_owner().
I wonder if it would be better to attach the blkio_cgroup to the anonymous page only when swap-out occurs. I mean, just put the blkio_cgroup_set_owner() hook in try_to_umap() in order to keep track of the IO generated by direct reclaim of anon memory. For all the other cases we can simply use the submitting task's context.
BTW, O_DIRECT is another case that is possible to optimize, because all the bios generated by direct IO occur in the same context of the current task.
-Andrea
Re: [PATCH] io-controller: Add io group reference handling for request
In reply to Andrea RighiQuoted 25 lines · Andrea Righi
Quoted 24 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)Hi Vivek,
Quoted 7 lines
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that
get_cgroup_from_bio()is much more slow thantask_cgroup()and need to lock/unlock_page_cgroup() inget_blkio_cgroup_id(), whiletask_cgroup()is rcu protected.
True.
Quoted 5 lines
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some
blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can usetask_cgroup().
Yes, may be in some cases we can avoid setting page owner. I will get to it once I have got functionality going well. In the mean time if you have a patch for it, it will be great.
Quoted 4 lines
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
Right now I am assuming that all the sync IO will belong to task submitting the bio hence use task_cgroup() for that. Only for async IO, I am trying to use page tracking functionality to determine the owner. Look at elv_bio_sync(bio).
You seem to be saying that there are cases where even for sync IO, we can't use submitting task's context and need to rely on page tracking functionlity? In case of getting page (read) from swap, will it not happen in the context of process who will take a page fault and initiate the swap read?
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 24 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 9 lines
}
block/elevator-fq.c +4
-0@@ -1462,20 +1462,27 @@ struct io_cgroup *get_iocg_from_bio(stru/* * Find the io group bio belongs to. * If "create" is set, io group is created if it is not already present. +* If "curr" is set, io group is information is searched for current +* task and not with the help of bio. +* +* FIXME: Can we assume that if bio is NULL then lookup group for current block/elevator-fq.c struct io_cgroup *get_iocg_from_bio(stru · line 1473
Quoted 10 lines
block/elevator-fq.c +3
-5+* task and not create extra function parameter ? * -* Note: There is a narrow window of race where a group is being freed -* by cgroup deletion path and some rq has slipped through in this group. -* Fix it. */ -struct io_group *io_get_io_group_bio (struct request_queue *q,struct bio *bio, -int create) +struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, +int create,int curr)Hi Vivek,
Quoted 2 lines
IIUC we can get rid of curr, and just determine iog from bio. If bio is not NULL, get iog from bio, otherwise get it from current task.
Consider also that get_cgroup_from_bio() is much more slow than task_cgroup() and need to lock/unlock_page_cgroup() in get_blkio_cgroup_id(), while task_cgroup() is rcu protected.
BTW another optimization could be to use the blkio-cgroup functionality only for dirty pages and cut out some blkio_set_owner(). For all the other cases IO always occurs in the same context of the current task, and you can use task_cgroup().
However, this is true only for page cache pages, for IO generated by anonymous pages (swap) you still need the page tracking functionality both for reads and writes.
-Andrea
Quoted 26 lines
Quoted 21 lines
block/elevator-fq.c +6
-1{struct cgroup *cgroup;struct io_group *iog;struct elv_fq_data *efqd = &q->elevator->efqd;rcu_read_lock (); - cgroup =get_cgroup_from_bio (bio); + +if (curr) + cgroup =task_cgroup (current, io_subsys_id); +else + cgroup =get_cgroup_from_bio (bio); +if (!cgroup) {if (create) iog = efqd->root_group;@@ -1500,7 +1507,7 @@ out:rcu_read_unlock ();return iog; }Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] io-controller: Add io group reference handling for request
In reply to Vivek GoyalVivek Goyal wrote:
...
Quoted 18 lines
Ok, here is the patch which gets rid of
rq->iogandrq->rlfields. Good to see some code and data structures trimming. It seems to be working fine for me.o Get rid of
rq->iogfield andrq->rlfields. request descriptor stores the pointer the the queue it belongs to (rq->ioq) and from the io queue one can determine the group queue belongs to hence request belongs to. Thanks to Nauman for the idea.o There are couple of places where
rq->ioqinformation is not present yet as request and queue are being setup. In those places "bio" is passed around as function argument to determine the group rq will go into. I did not pass "iog" as function argument becuase when memory is scarce, we can release queue lock and sleep to wait for memory to become available and once we wake up, it is possible that io group is gone. Passing bio around helps that one shall have to remap bio to right group after waking up.
Quoted 3 lines
o Got rid of
io_lookup_io_group_current()function and merged it withio_get_io_group()to also take care of looking for group using current task info and not from bio.
Hi Vivek,
This patch gets rid of "curr" from io_get_io_group, and seems to be working fine for me.
--- block/cfq-iosched.c | 12 ++++++------ block/elevator-fq.c | 16 ++++++++-------- block/elevator-fq.h | 2 +- 3 files changed, 15 insertions(+), 15 deletions(-)
block/cfq-iosched.c +6 -6
@@ -196,7 +196,7 @@ static struct cfq_queue *cic_bio_to_cfqq(struct cfq_data *cfqd,* async bio tracking is enabled and we are not caching * async queue pointer in cic. */ - iog =io_get_io_group (cfqd->queue, bio,0 ,0 ); + iog =io_get_io_group (cfqd->queue, bio,0 );if (!iog) {/* * May be this is first rq/bio and io group has not
@@ -1294,7 +1294,7 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)spin_lock_irqsave (q->queue_lock, flags); - iog =io_get_io_group (q,NULL ,0 ,1 ); + iog =io_get_io_group (q,NULL ,0 );if (async_cfqq !=NULL ) { __iog =cfqq_to_io_group (async_cfqq);
@@ -1346,9 +1346,9 @@ retry:* back. */ if (bio) - iog =io_get_io_group (q, bio,1 ,0 ); + iog =io_get_io_group (q, bio,1 );else - iog =io_get_io_group (q,NULL ,1 ,1 ); + iog =io_get_io_group (q,NULL ,1 ); cic =cfq_cic_lookup (cfqd, ioc);/* cic always exists here */
@@ -1469,9 +1469,9 @@ cfq_get_queue(struct cfq_data *cfqd, struct bio *bio, int is_sync,struct io_group *iog =NULL ;if (bio) - iog =io_get_io_group (cfqd->queue, bio,1 ,0 ); + iog =io_get_io_group (cfqd->queue, bio,1 );else - iog =io_get_io_group (cfqd->queue,NULL ,1 ,1 ); + iog =io_get_io_group (cfqd->queue,NULL ,1 );if (!is_sync) { async_cfqq = io_group_async_queue_prio(iog, ioprio_class,
block/elevator-fq.c +8 -8
@@ -1006,7 +1006,7 @@ struct request_list *io_group_get_request_list(struct request_queue *q, {struct io_group *iog; - iog =io_get_io_group (q, bio,1 ,0 ); + iog =io_get_io_group (q, bio,1 );BUG_ON (!iog);return &iog->rl; }
@@ -1470,7 +1470,7 @@ struct io_cgroup *get_iocg_from_bio(struct bio *bio)* */ struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, -int create,int curr) +int create) {struct cgroup *cgroup;struct io_group *iog;
@@ -1478,7 +1478,7 @@ struct io_group *io_get_io_group(struct request_queue *q, struct bio *bio,rcu_read_lock (); -if (curr) +if (!bio) cgroup =task_cgroup (current, io_subsys_id);else cgroup =get_cgroup_from_bio (bio);
@@ -1959,7 +1959,7 @@ int io_group_allow_merge(struct request *rq, struct bio *bio)return 1 ;/* Determine the io group of the bio submitting task */ - iog =io_get_io_group (q, bio,0 ,0 ); + iog =io_get_io_group (q, bio,0 );if (!iog) {/* May be task belongs to a differet cgroup for which io * group has not been setup yet. */
@@ -2000,9 +2000,9 @@ int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq, retry:/* Determine the io group request belongs to */ if (bio) - iog =io_get_io_group (q, bio,1 ,0 ); + iog =io_get_io_group (q, bio,1 );else - iog =io_get_io_group (q, bio,1 ,1 ); + iog =io_get_io_group (q,NULL ,1 );BUG_ON (!iog);
@@ -2098,7 +2098,7 @@ struct io_queue *elv_lookup_ioq_bio(struct request_queue *q, struct bio *bio)struct io_group *iog;/* lookup the io group and io queue of the bio submitting task */ - iog =io_get_io_group (q, bio,0 ,0 ); + iog =io_get_io_group (q, bio,0 );if (!iog) {/* May be bio belongs to a cgroup for which io group has * not been setup yet. */
@@ -2159,7 +2159,7 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q)EXPORT_SYMBOL (io_lookup_io_group_current);struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, -int create,int curr) +int create) {return q->elevator->efqd.root_group; }
block/elevator-fq.h +1 -1
@@ -657,7 +657,7 @@ extern void io_group_set_async_queue(struct io_group *iog, int ioprio_class, int ioprio, struct io_queue *ioq);extern struct io_group *io_lookup_io_group_current (struct request_queue *q);extern struct io_group *io_get_io_group (struct request_queue *q, -struct bio *bio,int create,int curr); +struct bio *bio,int create);extern int elv_nr_busy_ioq (struct elevator_queue *e);extern int elv_nr_busy_rt_ioq (struct elevator_queue *e);extern struct io_queue *elv_alloc_ioq (struct request_queue *q,gfp_t gfp_mask);
Signature · 1 line
1.5.4.rc3
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 20 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 18 lines
Ok, here is the patch which gets rid of
rq->iogandrq->rlfields. Good to see some code and data structures trimming. It seems to be working fine for me.o Get rid of
rq->iogfield andrq->rlfields. request descriptor stores the pointer the the queue it belongs to (rq->ioq) and from the io queue one can determine the group queue belongs to hence request belongs to. Thanks to Nauman for the idea.o There are couple of places where
rq->ioqinformation is not present yet as request and queue are being setup. In those places "bio" is passed around as function argument to determine the group rq will go into. I did not pass "iog" as function argument becuase when memory is scarce, we can release queue lock and sleep to wait for memory to become available and once we wake up, it is possible that io group is gone. Passing bio around helps that one shall have to remap bio to right group after waking up.
Quoted 9 lines
Quoted 3 lines
o Got rid of
io_lookup_io_group_current()function and merged it withio_get_io_group()to also take care of looking for group using current task info and not from bio.Hi Vivek,
This patch gets rid of "curr" from
io_get_io_group, and seems to be working fine for me.
Thanks Gui. Can't think of a reason why "curr" should be a separate function argument. I could only find blk_get_request() to be calling io_get_io_group() without any bio passed. I think even in that case determining the group from task should not hurt, I guess.
I will apply the patch. Couple of comments inline.
Quoted 30 lines
Signed-off-by Gui Jianfeng <[email protected]>--- block/cfq-iosched.c | 12 ++++++------ block/elevator-fq.c | 16 ++++++++-------- block/elevator-fq.h | 2 +- 3 files changed, 15 insertions(+), 15 deletions(-)block/cfq-iosched.c +2
-2@@ -196,7 +196,7 @@ static struct cfq_queue *cic_bio_to_cfqq(struct cfq_data *cfqd,* async bio tracking is enabled and we are not caching * async queue pointer in cic. */ - iog =io_get_io_group (cfqd->queue, bio,0 ,0 ); + iog =io_get_io_group (cfqd->queue, bio,0 );if (!iog) {/* * May be this is first rq/bio and io group has not @@ -1294,7 +1294,7 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)spin_lock_irqsave (q->queue_lock, flags); - iog =io_get_io_group (q,NULL ,0 ,1 ); + iog =io_get_io_group (q,NULL ,0 );if (async_cfqq !=NULL ) { __iog =cfqq_to_io_group (async_cfqq);
block/cfq-iosched.c retry: · line 1352
Quoted 9 lines
block/cfq-iosched.c +2
-2@@ -1346,9 +1346,9 @@ retry:* back. */ if (bio) - iog =io_get_io_group (q, bio,1 ,0 ); + iog =io_get_io_group (q, bio,1 );else - iog =io_get_io_group (q,NULL ,1 ,1 ); + iog =io_get_io_group (q,NULL ,1 );
Can we now change above to single statement
io_get_io_group(q, bio, 1);
if bio is present, we will determine the group from that otherwise from curret task context.
block/cfq-iosched.c cfq_get_queue(struct cfq_data *cfqd, struct bio *bio, int is_sync, · line 1475
Quoted 12 lines
block/cfq-iosched.c +2
-2cic =cfq_cic_lookup (cfqd, ioc);/* cic always exists here */ @@ -1469,9 +1469,9 @@ cfq_get_queue(struct cfq_data *cfqd, struct bio *bio, int is_sync,struct io_group *iog =NULL ;if (bio) - iog =io_get_io_group (cfqd->queue, bio,1 ,0 ); + iog =io_get_io_group (cfqd->queue, bio,1 );else - iog =io_get_io_group (cfqd->queue,NULL ,1 ,1 ); + iog =io_get_io_group (cfqd->queue,NULL ,1 );
Same here. Get rid of "if" condition.
Quoted 43 lines
block/cfq-iosched.c +0
-0if (!is_sync) { async_cfqq = io_group_async_queue_prio(iog, ioprio_class,block/elevator-fq.c +4
-4@@ -1006,7 +1006,7 @@ struct request_list *io_group_get_request_list(struct request_queue *q, {struct io_group *iog; - iog =io_get_io_group (q, bio,1 ,0 ); + iog =io_get_io_group (q, bio,1 );BUG_ON (!iog);return &iog->rl; }@@ -1470,7 +1470,7 @@ struct io_cgroup *get_iocg_from_bio(struct bio *bio)* */ struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, -int create,int curr) +int create) {struct cgroup *cgroup;struct io_group *iog;@@ -1478,7 +1478,7 @@ struct io_group *io_get_io_group(struct request_queue *q, struct bio *bio,rcu_read_lock (); -if (curr) +if (!bio) cgroup =task_cgroup (current, io_subsys_id);else cgroup =get_cgroup_from_bio (bio);@@ -1959,7 +1959,7 @@ int io_group_allow_merge(struct request *rq, struct bio *bio)return 1 ;/* Determine the io group of the bio submitting task */ - iog =io_get_io_group (q, bio,0 ,0 ); + iog =io_get_io_group (q, bio,0 );if (!iog) {/* May be task belongs to a differet cgroup for which io * group has not been setup yet. */
block/elevator-fq.c int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq, · line 2006
Quoted 9 lines
block/elevator-fq.c +2
-2@@ -2000,9 +2000,9 @@ int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq, retry:/* Determine the io group request belongs to */ if (bio) - iog =io_get_io_group (q, bio,1 ,0 ); + iog =io_get_io_group (q, bio,1 );else - iog =io_get_io_group (q, bio,1 ,1 ); + iog =io_get_io_group (q,NULL ,1 );
"if" not required now.
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 20 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 18 lines
Ok, here is the patch which gets rid of
rq->iogandrq->rlfields. Good to see some code and data structures trimming. It seems to be working fine for me.o Get rid of
rq->iogfield andrq->rlfields. request descriptor stores the pointer the the queue it belongs to (rq->ioq) and from the io queue one can determine the group queue belongs to hence request belongs to. Thanks to Nauman for the idea.o There are couple of places where
rq->ioqinformation is not present yet as request and queue are being setup. In those places "bio" is passed around as function argument to determine the group rq will go into. I did not pass "iog" as function argument becuase when memory is scarce, we can release queue lock and sleep to wait for memory to become available and once we wake up, it is possible that io group is gone. Passing bio around helps that one shall have to remap bio to right group after waking up.
Quoted 9 lines
Quoted 3 lines
o Got rid of
io_lookup_io_group_current()function and merged it withio_get_io_group()to also take care of looking for group using current task info and not from bio.Hi Vivek,
This patch gets rid of "curr" from
io_get_io_group, and seems to be working fine for me.
Thanks Gui. Can't think of a reason why "curr" should be a separate function argument. I could only find blk_get_request() to be calling io_get_io_group() without any bio passed. I think even in that case determining the group from task should not hurt, I guess.
I will apply the patch. Couple of comments inline.
Quoted 30 lines
Signed-off-by Gui Jianfeng <[email protected]>--- block/cfq-iosched.c | 12 ++++++------ block/elevator-fq.c | 16 ++++++++-------- block/elevator-fq.h | 2 +- 3 files changed, 15 insertions(+), 15 deletions(-)block/cfq-iosched.c +2
-2@@ -196,7 +196,7 @@ static struct cfq_queue *cic_bio_to_cfqq(struct cfq_data *cfqd,* async bio tracking is enabled and we are not caching * async queue pointer in cic. */ - iog =io_get_io_group (cfqd->queue, bio,0 ,0 ); + iog =io_get_io_group (cfqd->queue, bio,0 );if (!iog) {/* * May be this is first rq/bio and io group has not @@ -1294,7 +1294,7 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)spin_lock_irqsave (q->queue_lock, flags); - iog =io_get_io_group (q,NULL ,0 ,1 ); + iog =io_get_io_group (q,NULL ,0 );if (async_cfqq !=NULL ) { __iog =cfqq_to_io_group (async_cfqq);
block/cfq-iosched.c retry: · line 1352
Quoted 9 lines
block/cfq-iosched.c +2
-2@@ -1346,9 +1346,9 @@ retry:* back. */ if (bio) - iog =io_get_io_group (q, bio,1 ,0 ); + iog =io_get_io_group (q, bio,1 );else - iog =io_get_io_group (q,NULL ,1 ,1 ); + iog =io_get_io_group (q,NULL ,1 );
Can we now change above to single statement
io_get_io_group(q, bio, 1);
if bio is present, we will determine the group from that otherwise from curret task context.
block/cfq-iosched.c cfq_get_queue(struct cfq_data *cfqd, struct bio *bio, int is_sync, · line 1475
Quoted 12 lines
block/cfq-iosched.c +2
-2cic =cfq_cic_lookup (cfqd, ioc);/* cic always exists here */ @@ -1469,9 +1469,9 @@ cfq_get_queue(struct cfq_data *cfqd, struct bio *bio, int is_sync,struct io_group *iog =NULL ;if (bio) - iog =io_get_io_group (cfqd->queue, bio,1 ,0 ); + iog =io_get_io_group (cfqd->queue, bio,1 );else - iog =io_get_io_group (cfqd->queue,NULL ,1 ,1 ); + iog =io_get_io_group (cfqd->queue,NULL ,1 );
Same here. Get rid of "if" condition.
Quoted 43 lines
block/cfq-iosched.c +0
-0if (!is_sync) { async_cfqq = io_group_async_queue_prio(iog, ioprio_class,block/elevator-fq.c +4
-4@@ -1006,7 +1006,7 @@ struct request_list *io_group_get_request_list(struct request_queue *q, {struct io_group *iog; - iog =io_get_io_group (q, bio,1 ,0 ); + iog =io_get_io_group (q, bio,1 );BUG_ON (!iog);return &iog->rl; }@@ -1470,7 +1470,7 @@ struct io_cgroup *get_iocg_from_bio(struct bio *bio)* */ struct io_group *io_get_io_group (struct request_queue *q,struct bio *bio, -int create,int curr) +int create) {struct cgroup *cgroup;struct io_group *iog;@@ -1478,7 +1478,7 @@ struct io_group *io_get_io_group(struct request_queue *q, struct bio *bio,rcu_read_lock (); -if (curr) +if (!bio) cgroup =task_cgroup (current, io_subsys_id);else cgroup =get_cgroup_from_bio (bio);@@ -1959,7 +1959,7 @@ int io_group_allow_merge(struct request *rq, struct bio *bio)return 1 ;/* Determine the io group of the bio submitting task */ - iog =io_get_io_group (q, bio,0 ,0 ); + iog =io_get_io_group (q, bio,0 );if (!iog) {/* May be task belongs to a differet cgroup for which io * group has not been setup yet. */
block/elevator-fq.c int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq, · line 2006
Quoted 9 lines
block/elevator-fq.c +2
-2@@ -2000,9 +2000,9 @@ int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq, retry:/* Determine the io group request belongs to */ if (bio) - iog =io_get_io_group (q, bio,1 ,0 ); + iog =io_get_io_group (q, bio,1 );else - iog =io_get_io_group (q, bio,1 ,1 ); + iog =io_get_io_group (q,NULL ,1 );
"if" not required now.
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 43 lines · Gui Jianfeng
Quoted 34 lines · Nauman Rafique
Quoted 23 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Hi Vivek,
This patch adds io group reference handling when allocating and removing a request.
Hi Gui,
Thanks for the patch. We were thinking that requests can take a reference on io queues and io queues can take a reference on io groups. That should make sure that io groups don't go away as long as active requests are present.
But there seems to be a small window while allocating the new request where request gets allocated from a group first and then later it is mapped to that group and queue is created. IOW, in
get_request_wait(), we allocate a request from a particular group and setrq->rl, then drop the queue lock and later callelv_set_request()which again maps the request to the group savesrq->iogand creates new queue. This window is troublesome because request can be mapped to a particular group at the time of allocation and duringset_request()it can go to a different group as queue lock was dropped and group might have disappeared.Quoted 4 lines
In this case probably it might make sense that request also takes a reference on groups. At the same time it looks too much that request takes a reference on queue as well as group object. Ideas are welcome on how to handle it...
IMHO a request being allocated on the wrong cgroup should not be a big problem as such. All it means is that the request descriptor was accounted to the wrong cgroup in this particular corner case. Please correct me if I am wrong.
Quoted 7 lines
We can also get rid of
rq->iogpointer too. What that means is that request is associated with ioq (rq->ioq), and we can useioq_to_io_group()function to get theio_group. So the request would only be indirectly associated with anio_groupi.e. the request is associated with anio_queueand theio_groupfor the request is theio_groupassociated withio_queue. Do you see any problems with that approach?
Quoted 3 lines
That sounds reasonable to get rid of
rq->iog, andrq->rlis also dead. Hope to see the patch soon. ;)
Ok, here is the patch which gets rid of rq->iog and rq->rl fields. Good to see some code and data structures trimming. It seems to be working fine for me.
o Get rid of rq->iog field and rq->rl fields. request descriptor stores the pointer the the queue it belongs to (rq->ioq) and from the io queue one can determine the group queue belongs to hence request belongs to. Thanks to Nauman for the idea.
o There are couple of places where rq->ioq information is not present yet as request and queue are being setup. In those places "bio" is passed around as function argument to determine the group rq will go into. I did not pass "iog" as function argument becuase when memory is scarce, we can release queue lock and sleep to wait for memory to become available and once we wake up, it is possible that io group is gone. Passing bio around helps that one shall have to remap bio to right group after waking up.
o Got rid of io_lookup_io_group_current() function and merged it with io_get_io_group() to also take care of looking for group using current task info and not from bio.
--- block/blk-core.c | 28 +++++++++------- block/cfq-iosched.c | 40 ++++++++++++++++-------- block/elevator-fq.c | 78 ++++++++++++++++++----------------------------- block/elevator-fq.h | 29 +++-------------- block/elevator.c | 6 +-- include/linux/blkdev.h | 16 ++++----- include/linux/elevator.h | 2 - 7 files changed, 91 insertions(+), 108 deletions(-)
include/linux/blkdev.h +8 -8
@@ -279,12 +279,6 @@ struct request {#ifdef CONFIG_ELV_FAIR_QUEUING /* io queue request belongs to */ struct io_queue *ioq; - -#ifdef CONFIG_GROUP_IOSCHED -/* io group request belongs to */ -struct io_group *iog; -struct request_list *rl; -#endif /* GROUP_IOSCHED */ #endif /* ELV_FAIR_QUEUING */ };
@@ -828,9 +822,15 @@ static inline struct request_list *rq_rl struct request *rq) {#ifdef CONFIG_GROUP_IOSCHED -return rq->rl; +struct io_group *iog; + +BUG_ON (!rq->ioq); + iog =ioq_to_io_group (rq->ioq); +BUG_ON (!iog); + +return &iog->rl;#else -return blk_get_request_list (q,NULL ); +return &q->rq;#endif }
block/elevator-fq.c +31 -47
164 lines not on this page · raw message
include/linux/elevator.h +1 -1
@@ -23,7 +23,7 @@ typedef struct request *(elevator_requestypedef void (elevator_completed_req_fn ) (struct request_queue *,struct request *);typedef int (elevator_may_queue_fn ) (struct request_queue *,int ); -typedef int (elevator_set_req_fn ) (struct request_queue *,struct request *,gfp_t ); +typedef int (elevator_set_req_fn ) (struct request_queue *,struct request *,struct bio *bio,gfp_t );typedef void (elevator_put_req_fn ) (struct request *);typedef void (elevator_activate_req_fn ) (struct request_queue *,struct request *);typedef void (elevator_deactivate_req_fn ) (struct request_queue *,struct request *);
block/cfq-iosched.c +28 -12
@@ -161,7 +161,7 @@ CFQ_CFQQ_FNS(coop); blk_add_trace_msg((cfqd)->queue, "cfq " fmt, ##args) static void cfq_dispatch_insert(struct request_queue *, struct request *); -static struct cfq_queue *cfq_get_queue(struct cfq_data *, struct io_group *iog, +static struct cfq_queue *cfq_get_queue(struct cfq_data *, struct bio *bio, int, struct io_context *, gfp_t); static struct cfq_io_context *cfq_cic_lookup(struct cfq_data *, struct io_context *);
@@ -196,7 +196,7 @@ static struct cfq_queue *cic_bio_to_cfqq* async bio tracking is enabled and we are not caching * async queue pointer in cic. */ - iog =io_get_io_group_bio (cfqd->queue, bio,0 ); + iog =io_get_io_group (cfqd->queue, bio,0 ,0 );if (!iog) {/* * May be this is first rq/bio and io group has not
@@ -1242,7 +1242,6 @@ static void changed_ioprio(struct io_con cfqq = cic->cfqq[BLK_RW_ASYNC ];if (cfqq) { -struct io_group *iog =io_lookup_io_group_current (q);struct cfq_queue *new_cfqq;/*
@@ -1259,7 +1258,7 @@ static void changed_ioprio(struct io_con* comes? Keeping it for the time being because existing cfq * code allocates the new queue immediately upon prio change */ - new_cfqq =cfq_get_queue (cfqd, iog,BLK_RW_ASYNC , cic->ioc, + new_cfqq =cfq_get_queue (cfqd,NULL ,BLK_RW_ASYNC , cic->ioc,GFP_ATOMIC );if (new_cfqq)cic_set_cfqq (cic, new_cfqq,BLK_RW_ASYNC );
@@ -1295,7 +1294,7 @@ static void changed_cgroup(struct io_conspin_lock_irqsave (q->queue_lock, flags); - iog =io_lookup_io_group_current (q); + iog =io_get_io_group (q,NULL ,0 ,1 );if (async_cfqq !=NULL ) { __iog =cfqq_to_io_group (async_cfqq);
@@ -1332,14 +1331,25 @@ static void cfq_ioc_set_cgroup(struct io#endif /* CONFIG_IOSCHED_CFQ_HIER */ static struct cfq_queue * -cfq_find_alloc_queue (struct cfq_data *cfqd,struct io_group *iog,int is_sync, +cfq_find_alloc_queue (struct cfq_data *cfqd,struct bio *bio,int is_sync,struct io_context *ioc,gfp_t gfp_mask) {struct cfq_queue *cfqq, *new_cfqq =NULL ;struct cfq_io_context *cic;struct request_queue *q = cfqd->queue;struct io_queue *ioq =NULL , *new_ioq =NULL ; +struct io_group *iog =NULL ; retry: +/* +* Note: Finding the io group again in case io group disappeared +* during the time we dropped the queue lock and acquired it +* back. +*/ +if (bio) + iog =io_get_io_group (q, bio,1 ,0 ); +else + iog =io_get_io_group (q,NULL ,1 ,1 ); + cic =cfq_cic_lookup (cfqd, ioc);/* cic always exists here */ cfqq =cic_to_cfqq (cic, is_sync);
@@ -1449,13 +1459,19 @@ out: }static struct cfq_queue * -cfq_get_queue (struct cfq_data *cfqd,struct io_group *iog,int is_sync, -struct io_context *ioc,gfp_t gfp_mask) +cfq_get_queue (struct cfq_data *cfqd,struct bio *bio,int is_sync, +struct io_context *ioc,gfp_t gfp_mask) {const int ioprio =task_ioprio (ioc);const int ioprio_class =task_ioprio_class (ioc);struct cfq_queue *async_cfqq =NULL ;struct cfq_queue *cfqq =NULL ; +struct io_group *iog =NULL ; + +if (bio) + iog =io_get_io_group (cfqd->queue, bio,1 ,0 ); +else + iog =io_get_io_group (cfqd->queue,NULL ,1 ,1 );if (!is_sync) { async_cfqq = io_group_async_queue_prio(iog, ioprio_class,
@@ -1464,7 +1480,7 @@ cfq_get_queue(struct cfq_data *cfqd, str }if (!cfqq) { - cfqq =cfq_find_alloc_queue (cfqd, iog, is_sync, ioc, gfp_mask); + cfqq =cfq_find_alloc_queue (cfqd, bio, is_sync, ioc, gfp_mask);if (!cfqq)return NULL ; }
@@ -1889,7 +1905,8 @@ static void cfq_put_request(struct reque* Allocate cfq data structures associated with this request. */ static int -cfq_set_request (struct request_queue *q,struct request *rq,gfp_t gfp_mask) +cfq_set_request (struct request_queue *q,struct request *rq,struct bio *bio, +gfp_t gfp_mask) {struct cfq_data *cfqd = q->elevator->elevator_data;struct cfq_io_context *cic;
@@ -1909,8 +1926,7 @@ cfq_set_request(struct request_queue *q, cfqq =cic_to_cfqq (cic, is_sync);if (!cfqq) { - cfqq =cfq_get_queue (cfqd,rq_iog (q, rq), is_sync, cic->ioc, - gfp_mask); + cfqq =cfq_get_queue (cfqd, bio, is_sync, cic->ioc, gfp_mask);if (!cfqq) goto queue_fail;
block/elevator.c +2 -4
@@ -972,17 +972,15 @@ int elv_set_request(struct request_queue {struct elevator_queue *e = q->elevator; -elv_fq_set_request_io_group (q, rq, bio); -/* * Optimization for noop, deadline and AS which maintain only single * ioq per io group */ if (elv_iosched_single_ioq (e)) -return elv_fq_set_request_ioq (q, rq, gfp_mask); +return elv_fq_set_request_ioq (q, rq, bio, gfp_mask);if (e->ops->elevator_set_req_fn) -return e->ops->elevator_set_req_fn (q, rq, gfp_mask); +return e->ops->elevator_set_req_fn (q, rq, bio, gfp_mask); rq->elevator_private =NULL ;return 0 ;
block/elevator-fq.h +5 -24
@@ -510,15 +510,13 @@ static inline struct io_group *ioq_to_io#ifdef CONFIG_GROUP_IOSCHED extern int io_group_allow_merge (struct request *rq,struct bio *bio); -extern void elv_fq_set_request_io_group (struct request_queue *q, -struct request *rq,struct bio *bio);static inline bfq_weight_t iog_weight (struct io_group *iog) {return iog->entity.weight; }extern int elv_fq_set_request_ioq (struct request_queue *q,struct request *rq, -gfp_t gfp_mask); +struct bio *bio,gfp_t gfp_mask);extern void elv_fq_unset_request_ioq (struct request_queue *q,struct request *rq);extern struct io_queue *elv_lookup_ioq_current (struct request_queue *q);
@@ -545,12 +543,6 @@ static inline void io_group_set_ioq(stru iog->ioq = ioq; } -static inline struct io_group *rq_iog (struct request_queue *q, -struct request *rq) -{ -return rq->iog; -} -static inline void elv_get_iog (struct io_group *iog) {atomic_inc (&iog->ref);
@@ -566,10 +558,6 @@ static inline int io_group_allow_merge(s* separately. Hence in case of non-hierarchical setup, nothing todo. */ static inline void io_disconnect_groups (struct elevator_queue *e) {} -static inline void elv_fq_set_request_io_group (struct request_queue *q, -struct request *rq,struct bio *bio) -{ -}static inline bfq_weight_t iog_weight (struct io_group *iog) {
@@ -588,7 +576,7 @@ static inline void io_group_set_ioq(stru }static inline int elv_fq_set_request_ioq (struct request_queue *q, -struct request *rq,gfp_t gfp_mask) +struct request *rq,struct bio *bio,gfp_t gfp_mask) {return 0 ; }
@@ -613,8 +601,6 @@ static inline void elv_get_iog(struct iostatic inline void elv_put_iog (struct io_group *iog) { } -extern struct io_group *rq_iog (struct request_queue *q,struct request *rq); -#endif /* GROUP_IOSCHED */ /* Functions used by blksysfs.c */
@@ -670,8 +656,8 @@ extern void *io_group_async_queue_prio(sextern void io_group_set_async_queue (struct io_group *iog,int ioprio_class,int ioprio,struct io_queue *ioq);extern struct io_group *io_lookup_io_group_current (struct request_queue *q); -extern struct io_group *io_get_io_group_bio (struct request_queue *q, -struct bio *bio,int create); +extern struct io_group *io_get_io_group (struct request_queue *q, +struct bio *bio,int create,int curr);extern int elv_nr_busy_ioq (struct elevator_queue *e);extern int elv_nr_busy_rt_ioq (struct elevator_queue *e);extern struct io_queue *elv_alloc_ioq (struct request_queue *q,gfp_t gfp_mask);
@@ -725,18 +711,13 @@ static inline void *elv_fq_select_ioq(streturn NULL ; } -static inline void elv_fq_set_request_io_group (struct request_queue *q, -struct request *rq,struct bio *bio) -{ -} -static inline int io_group_allow_merge (struct request *rq,struct bio *bio) {return 1 ; }static inline int elv_fq_set_request_ioq (struct request_queue *q, -struct request *rq,gfp_t gfp_mask) +struct request *rq,struct bio *bio,gfp_t gfp_mask) {return 0 ; }
block/blk-core.c +16 -12
@@ -736,8 +736,22 @@ static void __freed_request(struct requestatic void freed_request (struct request_queue *q,int sync,int priv,struct request_list *rl) { -BUG_ON (!rl->count[sync]); - rl->count[sync]--; +/* There is a window during request allocation where request is +* mapped to one group but by the time a queue for the group is +* allocated, it is possible that original cgroup/io group has been +* deleted and now io queue is allocated in a different group (root) +* altogether. +* +* One solution to the problem is that rq should take io group +* reference. But it looks too much to do that to solve this issue. +* The only side affect to the hard to hit issue seems to be that +* we will try to decrement the rl->count for a request list which +* did not allocate that request. Chcek for rl->count going less than +* zero and do not decrement it if that's the case. +*/ + +if (rl->count[sync] >0 ) + rl->count[sync]--;BUG_ON (!q->rq_data.count[sync]); q->rq_data.count[sync]--;
@@ -841,16 +855,6 @@ static struct request *get_request(struc rq =blk_alloc_request (q, bio, rw_flags, priv, gfp_mask); -#ifdef CONFIG_GROUP_IOSCHED -if (rq) { -/* -* TODO. Implement group reference counting and take the -* reference to the group to make sure group hence request -* list does not go away till rq finishes. -*/ - rq->rl = rl; - } -#endif if (unlikely (!rq)) {/* * Allocation failed presumably due to memory. Undo anything
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] io-controller: Add io group reference handling for request
Quoted 24 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Hi Vivek,
This patch adds io group reference handling when allocating and removing a request.
Hi Gui,
Thanks for the patch. We were thinking that requests can take a reference on io queues and io queues can take a reference on io groups. That should make sure that io groups don't go away as long as active requests are present.
But there seems to be a small window while allocating the new request where request gets allocated from a group first and then later it is mapped to that group and queue is created. IOW, in
get_request_wait(), we allocate a request from a particular group and setrq->rl, then drop the queue lock and later callelv_set_request()which again maps the request to the group savesrq->iogand creates new queue. This window is troublesome because request can be mapped to a particular group at the time of allocation and duringset_request()it can go to a different group as queue lock was dropped and group might have disappeared.
Quoted 4 lines
In this case probably it might make sense that request also takes a reference on groups. At the same time it looks too much that request takes a reference on queue as well as group object. Ideas are welcome on how to handle it...
IMHO a request being allocated on the wrong cgroup should not be a big problem as such. All it means is that the request descriptor was accounted to the wrong cgroup in this particular corner case. Please correct me if I am wrong.
We can also get rid of rq->iog pointer too. What that means is that request is associated with ioq (rq->ioq), and we can use ioq_to_io_group() function to get the io_group. So the request would only be indirectly associated with an io_group i.e. the request is associated with an io_queue and the io_group for the request is the io_group associated with io_queue. Do you see any problems with that approach?
Thanks.
--
Nauman
Quoted 79 lines
Thanks
VivekQuoted 74 lines
Signed-off-by Gui Jianfeng <[email protected]>---
elevator-fq.c | 15 ++++++++++++++-
elevator-fq.h | 5 +++++
elevator.c | 2 ++3 files changed, 21 insertions(+), 1 deletion(-)block/elevator-fq.c +14
-1@@ -1968,11 +1968,24 @@ void elv_fq_set_request_io_group(struct request_queue *q, struct request *rq, spin_unlock_irqrestore(q->queue_lock, flags); BUG_ON(!iog); - /* Store iog in rq. TODO: take care of referencing */ + elv_get_iog(iog); rq->iog = iog; } /* +* This request has been serviced. Clean up iog info and drop the reference. +*/ +void elv_fq_unset_request_io_group (struct request *rq) +{ + struct io_group *iog = rq->iog; + + if (iog) { + rq->iog = NULL; + elv_put_iog(iog); + } +} + +/* * Find/Create the io queue the rq should go in. This is an optimization * for the io schedulers (noop, deadline and AS) which maintain only single * io queue per cgroup. In this case common layer can just maintain a block/elevator-fq.h +5
-0@@ -512,6 +512,7 @@ static inline struct io_group *ioq_to_io_group(struct io_queue *ioq) extern int io_group_allow_merge(struct request *rq, struct bio *bio); extern void elv_fq_set_request_io_group(struct request_queue *q, struct request *rq, struct bio *bio); +extern void elv_fq_unset_request_io_group (struct request *rq); static inline bfq_weight_t iog_weight(struct io_group *iog) { return iog->entity.weight;@@ -571,6 +572,10 @@ static inline void elv_fq_set_request_io_group(struct request_queue *q, { } +static inline void elv_fq_unset_request_io_group (struct request *rq) +{ +} + static inline bfq_weight_t iog_weight(struct io_group *iog) { /* Just root group is present and weight is immaterial. */block/elevator.c +2
-0@@ -992,6 +992,8 @@ void elv_put_request(struct request_queue *q, struct request *rq) { struct elevator_queue *e = q->elevator; + elv_fq_unset_request_io_group(rq); + /** Optimization for noop, deadline and AS which maintain only single * ioq per io group
PATCH 1/1 IO Controller: Add per-device weight and ioprio_class handling
In reply to Vivek GoyalHi Vivek,
This patch enables per-cgroup per-device weight and ioprio_class handling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight and ioprio_class for each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.
You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on /dev/hdb in this cgroup
# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2
Configure weight=500 ioprio_class=1 on /dev/hda in this cgroup
# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2
Remove the policy for /dev/hda in this cgroup
# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2
--- block/elevator-fq.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 11 +++ 2 files changed, 245 insertions(+), 5 deletions(-)
block/elevator-fq.c +234 -5
@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;
@@ -1045,12 +1048,30 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +void *key) {struct io_entity *entity = &iog->entity; +struct policy_node *pn; + +spin_lock_irq (&iocg->lock); + pn =policy_search_node (iocg, key); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irq (&iocg->lock); - entity->weight = entity->new_weight = iocg->weight; - entity->ioprio_class = entity->new_ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sched_data = &iog->sched_data; }
@@ -1263,7 +1284,7 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,atomic_set (&iog->ref,0 ); iog->deleting =0 ; -io_group_init_entity (iocg, iog); +io_group_init_entity (iocg, iog, key); iog->my_entity = &iog->entity;#ifdef CONFIG_DEBUG_GROUP_IOSCHED iog->iocg_id =css_id (&iocg->css);
@@ -1549,8 +1570,208 @@ struct io_group *io_alloc_root_group(struct request_queue *q,return iog; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->list)) + goto out; + +seq_printf (m,"dev weight class\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->list, node) { +seq_printf (m,"%s %lu %lu\n" , pn->dev_name, + pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct policy_node *pn) +{ +list_add (&pn->node, &iocg->list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key) +{ +struct policy_node *pn; + +if (list_empty (&iocg->list)) +return NULL ; + +list_for_each_entry (pn, &iocg->list, node) { +if (pn->key == key) +return pn; + } + +return NULL ; +} + +static void *devname_to_efqd (const char *buf) +{ +struct block_device *bdev; +void *key =NULL ; +struct gendisk *disk; +int part; + + bdev =lookup_bdev (buf); +if (IS_ERR (bdev)) +return NULL ; + + disk =get_gendisk (bdev->bd_dev, &part); + key = (void *)&disk->queue->elevator->efqd; +bdput (bdev); + +return key; +} + +static int policy_parse_and_set (char *buf,struct policy_node *newpn) +{ +char *s[3 ]; +char *p; +int ret; +int i =0 ; + +memset (s,0 ,sizeof (s)); +while (i <ARRAY_SIZE (s)) { + p =strsep (&buf,":" ); +if (!p) +break ; +if (!*p) +continue ; + s[i++] = p; + } + + newpn->key =devname_to_efqd (s[0 ]); +if (!newpn->key) +return -EINVAL ; + +strcpy (newpn->dev_name, s[0 ]); + + ret =strict_strtoul (s[1 ],10 , &newpn->weight); +if (ret || newpn->weight >WEIGHT_MAX ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &newpn->ioprio_class); +if (ret || newpn->ioprio_class <IOPRIO_CLASS_RT || + newpn->ioprio_class >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + +return 0 ; +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->key); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->key == newpn->key) { +if (newpn->weight) { + iog->entity.new_weight = newpn->weight; + iog->entity.new_ioprio_class = + newpn->ioprio_class; +/* +* iog weight and ioprio_class updating +* actually happens if ioprio_changed is set. +* So ensure ioprio_changed is not set until +* new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.new_weight = iocg->weight; + iog->entity.new_ioprio_class = + iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } + } + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +struct cftype bfqio_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,
@@ -1592,6 +1813,7 @@ struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_DEFAULT_GRP_WEIGHT ; iocg->ioprio_class =IO_DEFAULT_GRP_CLASS ; +INIT_LIST_HEAD (&iocg->list);return &iocg->css; }
@@ -1750,6 +1972,7 @@ void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)unsigned long flags, flags1;int queue_lock_held =0 ;struct elv_fq_data *efqd; +struct policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained
@@ -1823,6 +2046,12 @@ locked:BUG_ON (!hlist_empty (&iocg->group_data));free_css_id (&io_subsys, &iocg->css); + +list_for_each_entry_safe (pn, pntmp, &iocg->list, node) { +policy_delete_node (pn); +kfree (pn); + } +kfree (iocg); }
@@ -2137,7 +2366,7 @@ void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)void bfq_init_entity (struct io_entity *entity,struct io_group *iog) { entity->ioprio = entity->new_ioprio; - entity->weight = entity->new_weight; + entity->weight = entity->new_weigh; entity->ioprio_class = entity->new_ioprio_class; entity->sched_data = &iog->sched_data; }
block/elevator-fq.h +11 -0
@@ -253,6 +253,14 @@ struct io_group {#endif }; +struct policy_node { +struct list_head node; +char dev_name[32 ]; +void *key; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup.
@@ -269,6 +277,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of policy_node */ +struct list_head list; +spinlock_t lock;struct hlist_head group_data; };
Signature · 1 line
1.5.4.rc3
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.
Quoted 4 lines
You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Thanks for the patch Gui. I will test it out and let you know how does it go.
Thanks
Vivek
Quoted 363 lines
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Configure weight=500 ioprio_class=1 on
/dev/hdain this cgroup# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2Remove the policy for
/dev/hdain this cgroup# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Signed-off-by Gui Jianfeng <[email protected]>--- block/elevator-fq.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 11 +++ 2 files changed, 245 insertions(+), 5 deletions(-)block/elevator-fq.c +234
-5@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -1045,12 +1048,30 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +void *key) {struct io_entity *entity = &iog->entity; +struct policy_node *pn; + +spin_lock_irq (&iocg->lock); + pn =policy_search_node (iocg, key); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irq (&iocg->lock); - entity->weight = entity->new_weight = iocg->weight; - entity->ioprio_class = entity->new_ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sched_data = &iog->sched_data; }@@ -1263,7 +1284,7 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,atomic_set (&iog->ref,0 ); iog->deleting =0 ; -io_group_init_entity (iocg, iog); +io_group_init_entity (iocg, iog, key); iog->my_entity = &iog->entity;#ifdef CONFIG_DEBUG_GROUP_IOSCHED iog->iocg_id =css_id (&iocg->css);@@ -1549,8 +1570,208 @@ struct io_group *io_alloc_root_group(struct request_queue *q,return iog; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->list)) + goto out; + +seq_printf (m,"dev weight class\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->list, node) { +seq_printf (m,"%s %lu %lu\n" , pn->dev_name, + pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct policy_node *pn) +{ +list_add (&pn->node, &iocg->list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key) +{ +struct policy_node *pn; + +if (list_empty (&iocg->list)) +return NULL ; + +list_for_each_entry (pn, &iocg->list, node) { +if (pn->key == key) +return pn; + } + +return NULL ; +} + +static void *devname_to_efqd (const char *buf) +{ +struct block_device *bdev; +void *key =NULL ; +struct gendisk *disk; +int part; + + bdev =lookup_bdev (buf); +if (IS_ERR (bdev)) +return NULL ; + + disk =get_gendisk (bdev->bd_dev, &part); + key = (void *)&disk->queue->elevator->efqd; +bdput (bdev); + +return key; +} + +static int policy_parse_and_set (char *buf,struct policy_node *newpn) +{ +char *s[3 ]; +char *p; +int ret; +int i =0 ; + +memset (s,0 ,sizeof (s)); +while (i <ARRAY_SIZE (s)) { + p =strsep (&buf,":" ); +if (!p) +break ; +if (!*p) +continue ; + s[i++] = p; + } + + newpn->key =devname_to_efqd (s[0 ]); +if (!newpn->key) +return -EINVAL ; + +strcpy (newpn->dev_name, s[0 ]); + + ret =strict_strtoul (s[1 ],10 , &newpn->weight); +if (ret || newpn->weight >WEIGHT_MAX ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &newpn->ioprio_class); +if (ret || newpn->ioprio_class <IOPRIO_CLASS_RT || + newpn->ioprio_class >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + +return 0 ; +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->key); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->key == newpn->key) { +if (newpn->weight) { + iog->entity.new_weight = newpn->weight; + iog->entity.new_ioprio_class = + newpn->ioprio_class; +/* +* iog weight and ioprio_class updating +* actually happens if ioprio_changed is set. +* So ensure ioprio_changed is not set until +* new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.new_weight = iocg->weight; + iog->entity.new_ioprio_class = + iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } + } + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +struct cftype bfqio_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,@@ -1592,6 +1813,7 @@ struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_DEFAULT_GRP_WEIGHT ; iocg->ioprio_class =IO_DEFAULT_GRP_CLASS ; +INIT_LIST_HEAD (&iocg->list);return &iocg->css; }@@ -1750,6 +1972,7 @@ void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)unsigned long flags, flags1;int queue_lock_held =0 ;struct elv_fq_data *efqd; +struct policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained @@ -1823,6 +2046,12 @@ locked:BUG_ON (!hlist_empty (&iocg->group_data));free_css_id (&io_subsys, &iocg->css); + +list_for_each_entry_safe (pn, pntmp, &iocg->list, node) { +policy_delete_node (pn); +kfree (pn); + } +kfree (iocg); }@@ -2137,7 +2366,7 @@ void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)void bfq_init_entity (struct io_entity *entity,struct io_group *iog) { entity->ioprio = entity->new_ioprio; - entity->weight = entity->new_weight; + entity->weight = entity->new_weigh; entity->ioprio_class = entity->new_ioprio_class; entity->sched_data = &iog->sched_data; }block/elevator-fq.h +11
-0@@ -253,6 +253,14 @@ struct io_group {#endif }; +struct policy_node { +struct list_head node; +char dev_name[32 ]; +void *key; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup. @@ -269,6 +277,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of policy_node */ +struct list_head list; +spinlock_t lock;struct hlist_head group_data; };Signature · 1 line
1.5.4.rc3
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Quoted 10 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.
Quoted 7 lines
Quoted 4 lines
You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Thanks for the patch Gui. I will test it out and let you know how does it go.
Hi Vivek,
I forgot to mention that this patch isn't tested through, just for showing the design. I'd like to test it soon.
Quoted 4 lines
Thanks
Vivek
Signature · 2 lines
Regards Gui Jianfeng
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Quoted 10 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.
Quoted 7 lines
Quoted 4 lines
You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Thanks for the patch Gui. I will test it out and let you know how does it go.
Hi Vivek,
I forgot to mention that this patch isn't tested through, just for showing the design. I'd like to test it soon.
Quoted 4 lines
Thanks
Vivek
Signature · 2 lines
Regards Gui Jianfeng
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.
Quoted 4 lines
You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Thanks for the patch Gui. I will test it out and let you know how does it go.
Thanks
Vivek
Quoted 363 lines
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Configure weight=500 ioprio_class=1 on
/dev/hdain this cgroup# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2Remove the policy for
/dev/hdain this cgroup# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Signed-off-by Gui Jianfeng <[email protected]>--- block/elevator-fq.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 11 +++ 2 files changed, 245 insertions(+), 5 deletions(-)block/elevator-fq.c +234
-5@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -1045,12 +1048,30 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +void *key) {struct io_entity *entity = &iog->entity; +struct policy_node *pn; + +spin_lock_irq (&iocg->lock); + pn =policy_search_node (iocg, key); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irq (&iocg->lock); - entity->weight = entity->new_weight = iocg->weight; - entity->ioprio_class = entity->new_ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sched_data = &iog->sched_data; }@@ -1263,7 +1284,7 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,atomic_set (&iog->ref,0 ); iog->deleting =0 ; -io_group_init_entity (iocg, iog); +io_group_init_entity (iocg, iog, key); iog->my_entity = &iog->entity;#ifdef CONFIG_DEBUG_GROUP_IOSCHED iog->iocg_id =css_id (&iocg->css);@@ -1549,8 +1570,208 @@ struct io_group *io_alloc_root_group(struct request_queue *q,return iog; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->list)) + goto out; + +seq_printf (m,"dev weight class\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->list, node) { +seq_printf (m,"%s %lu %lu\n" , pn->dev_name, + pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct policy_node *pn) +{ +list_add (&pn->node, &iocg->list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key) +{ +struct policy_node *pn; + +if (list_empty (&iocg->list)) +return NULL ; + +list_for_each_entry (pn, &iocg->list, node) { +if (pn->key == key) +return pn; + } + +return NULL ; +} + +static void *devname_to_efqd (const char *buf) +{ +struct block_device *bdev; +void *key =NULL ; +struct gendisk *disk; +int part; + + bdev =lookup_bdev (buf); +if (IS_ERR (bdev)) +return NULL ; + + disk =get_gendisk (bdev->bd_dev, &part); + key = (void *)&disk->queue->elevator->efqd; +bdput (bdev); + +return key; +} + +static int policy_parse_and_set (char *buf,struct policy_node *newpn) +{ +char *s[3 ]; +char *p; +int ret; +int i =0 ; + +memset (s,0 ,sizeof (s)); +while (i <ARRAY_SIZE (s)) { + p =strsep (&buf,":" ); +if (!p) +break ; +if (!*p) +continue ; + s[i++] = p; + } + + newpn->key =devname_to_efqd (s[0 ]); +if (!newpn->key) +return -EINVAL ; + +strcpy (newpn->dev_name, s[0 ]); + + ret =strict_strtoul (s[1 ],10 , &newpn->weight); +if (ret || newpn->weight >WEIGHT_MAX ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &newpn->ioprio_class); +if (ret || newpn->ioprio_class <IOPRIO_CLASS_RT || + newpn->ioprio_class >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + +return 0 ; +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->key); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->key == newpn->key) { +if (newpn->weight) { + iog->entity.new_weight = newpn->weight; + iog->entity.new_ioprio_class = + newpn->ioprio_class; +/* +* iog weight and ioprio_class updating +* actually happens if ioprio_changed is set. +* So ensure ioprio_changed is not set until +* new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.new_weight = iocg->weight; + iog->entity.new_ioprio_class = + iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } + } + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +struct cftype bfqio_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,@@ -1592,6 +1813,7 @@ struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_DEFAULT_GRP_WEIGHT ; iocg->ioprio_class =IO_DEFAULT_GRP_CLASS ; +INIT_LIST_HEAD (&iocg->list);return &iocg->css; }@@ -1750,6 +1972,7 @@ void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)unsigned long flags, flags1;int queue_lock_held =0 ;struct elv_fq_data *efqd; +struct policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained @@ -1823,6 +2046,12 @@ locked:BUG_ON (!hlist_empty (&iocg->group_data));free_css_id (&io_subsys, &iocg->css); + +list_for_each_entry_safe (pn, pntmp, &iocg->list, node) { +policy_delete_node (pn); +kfree (pn); + } +kfree (iocg); }@@ -2137,7 +2366,7 @@ void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)void bfq_init_entity (struct io_entity *entity,struct io_group *iog) { entity->ioprio = entity->new_ioprio; - entity->weight = entity->new_weight; + entity->weight = entity->new_weigh; entity->ioprio_class = entity->new_ioprio_class; entity->sched_data = &iog->sched_data; }block/elevator-fq.h +11
-0@@ -253,6 +253,14 @@ struct io_group {#endif }; +struct policy_node { +struct list_head node; +char dev_name[32 ]; +void *key; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup. @@ -269,6 +277,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of policy_node */ +struct list_head list; +spinlock_t lock;struct hlist_head group_data; };Signature · 1 line
1.5.4.rc3
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to a message not in the archiveOn Wed, May 13, 2009 at 10:00:21AM +0800, Gui Jianfeng wrote:
[..]
Quoted 14 lines
block/elevator-fq.c +1
-1@@ -2137,7 +2366,7 @@ void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)void bfq_init_entity (struct io_entity *entity,struct io_group *iog) { entity->ioprio = entity->new_ioprio; - entity->weight = entity->new_weight; + entity->weight = entity->new_weigh; entity->ioprio_class = entity->new_ioprio_class; entity->sched_data = &iog->sched_data; }block/elevator-fq.h +0
-0
block/elevator-fq.h struct io_group { · line 256
Quoted 4 lines
block/elevator-fq.h +1
-0@@ -253,6 +253,14 @@ struct io_group {#endif }; +struct policy_node {
Would "io_policy_node" be better?
Quoted 11 lines
block/elevator-fq.h +7
-0+struct list_head node; +char dev_name[32 ]; +void *key; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup.
block/elevator-fq.h struct io_cgroup { · line 282
Quoted 6 lines
block/elevator-fq.h +3
-0@@ -269,6 +277,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of policy_node */ +struct list_head list; +
How about "struct list_head policy_list" or "struct list_head io_policy"?
Thanks
Vivek
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to a message not in the archiveQuoted 71 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Configure weight=500 ioprio_class=1 on
/dev/hdain this cgroup# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2Remove the policy for
/dev/hdain this cgroup# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Signed-off-by Gui Jianfeng <[email protected]>--- block/elevator-fq.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 11 +++ 2 files changed, 245 insertions(+), 5 deletions(-)block/elevator-fq.c +14
-1@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -1045,12 +1048,30 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +void *key) {struct io_entity *entity = &iog->entity; +struct policy_node *pn; + +spin_lock_irq (&iocg->lock); + pn =policy_search_node (iocg, key); +if (pn) { + entity->weight = pn->weight;
block/elevator-fq.c struct io_group *io_lookup_io_group_current(struct request_queue *q) · line 1073
Quoted 10 lines
block/elevator-fq.c +10
-0+ entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irq (&iocg->lock);
Hi Gui,
It might make sense to also store the device name or device major and minor number in io_group while creating the io group. This will help us to display io.disk_time and io.disk_sector statistics per device instead of aggregate.
I am attaching a patch I was playing around with to display per device statistics instead of aggregate one. So if user has specified the per device rule.
Thanks
Vivek
o Currently the statistics exported through cgroup are aggregate of statistics on all devices for that cgroup. Instead of aggregate, make these per device.
o Also export another statistics io.disk_dequeue. This keeps a count of how many times a particular group got out of race for the disk. This is a debugging aid to keep a track how often we could create continuously backlogged queues.
--- block/elevator-fq.c | 127 +++++++++++++++++++++++++++++++++------------------- block/elevator-fq.h | 3 + 2 files changed, 85 insertions(+), 45 deletions(-)
block/elevator-fq.h +3 -0
@@ -250,6 +250,9 @@ struct io_group {#ifdef CONFIG_DEBUG_GROUP_IOSCHED unsigned short iocg_id; +dev_t dev; +/* How many times this group has been removed from active tree */ +unsigned long dequeue;#endif };
block/elevator-fq.c +82 -45
232 lines not on this page · raw message
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to a message not in the archiveQuoted 72 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Configure weight=500 ioprio_class=1 on
/dev/hdain this cgroup# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2Remove the policy for
/dev/hdain this cgroup# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Signed-off-by Gui Jianfeng <[email protected]>--- block/elevator-fq.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 11 +++ 2 files changed, 245 insertions(+), 5 deletions(-)block/elevator-fq.c +15
-1@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -1045,12 +1048,30 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +void *key) {struct io_entity *entity = &iog->entity; +struct policy_node *pn; + +spin_lock_irq (&iocg->lock); + pn =policy_search_node (iocg, key); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight;
block/elevator-fq.c struct io_group *io_lookup_io_group_current(struct request_queue *q) · line 1074
Quoted 10 lines
block/elevator-fq.c +9
-0+ entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irq (&iocg->lock);
I think we need to use spin_lock_irqsave() and spin_lock_irqrestore() version above because it can be called with request queue lock held and we don't want to enable the interrupts unconditionally here.
I hit following lock validator warning.
[ 81.521242] =================================
[ 81.522127] [ INFO: inconsistent lock state ]
[ 81.522127] 2.6.30-rc4-ioc #47
[ 81.522127] ---------------------------------
[ 81.522127] inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage.
[ 81.522127] io-group-bw-tes/4138 [HC0[0]:SC0[0]:HE1:SE1] takes:
[ 81.522127] (&q->__queue_lock){+.?...}, at: [<ffffffff811d7b2e>] __make_request+0x35/0x396
[ 81.522127] {IN-SOFTIRQ-W} state was registered at:
[ 81.522127] [<ffffffffffffffff>] 0xffffffffffffffff
[ 81.522127] irq event stamp: 1006
[ 81.522127] hardirqs last enabled at (1005): [<ffffffff810c1198>] kmem_cache_alloc+0x9d/0x105
[ 81.522127] hardirqs last disabled at (1006): [<ffffffff8150343f>] _spin_lock_irq+0x12/0x3e
[ 81.522127] softirqs last enabled at (286): [<ffffffff81042039>] __do_softirq+0x17a/0x187
[ 81.522127] softirqs last disabled at (271): [<ffffffff8100ccfc>] call_softirq+0x1c/0x34
[ 81.522127]
[ 81.522127] other info that might help us debug this:
[ 81.522127] 3 locks held by io-group-bw-tes/4138:
[ 81.522127] #0: (&type->i_mutex_dir_key#4){+.+.+.}, at: [<ffffffff810cfd2c>] do_lookup+0x82/0x15f
[ 81.522127] #1: (&q->__queue_lock){+.?...}, at: [<ffffffff811d7b2e>] __make_request+0x35/0x396
[ 81.522127] #2: (rcu_read_lock){.+.+..}, at: [<ffffffff811e55bb>] __rcu_read_lock+0x0/0x30
[ 81.522127]
[ 81.522127] stack backtrace:
[ 81.522127] Pid: 4138, comm: io-group-bw-tes Not tainted 2.6.30-rc4-ioc #47
[ 81.522127] Call Trace:
[ 81.522127] [<ffffffff8105edad>] valid_state+0x17c/0x18f
[ 81.522127] [<ffffffff8105eb8a>] ? check_usage_backwards+0x0/0x52
[ 81.522127] [<ffffffff8105ee9b>] mark_lock+0xdb/0x1ff
[ 81.522127] [<ffffffff8105f00c>] mark_held_locks+0x4d/0x6b
[ 81.522127] [<ffffffff8150331a>] ? _spin_unlock_irq+0x2b/0x31
[ 81.522127] [<ffffffff8105f13e>] trace_hardirqs_on_caller+0x114/0x138
[ 81.522127] [<ffffffff8105f16f>] trace_hardirqs_on+0xd/0xf
[ 81.522127] [<ffffffff8150331a>] _spin_unlock_irq+0x2b/0x31
[ 81.522127] [<ffffffff811e5534>] ? io_group_init_entity+0x2a/0xb1
[ 81.522127] [<ffffffff811e5597>] io_group_init_entity+0x8d/0xb1
[ 81.522127] [<ffffffff811e688e>] ? io_group_chain_alloc+0x49/0x167
[ 81.522127] [<ffffffff811e68fe>] io_group_chain_alloc+0xb9/0x167
[ 81.522127] [<ffffffff811e6a04>] io_find_alloc_group+0x58/0x85
[ 81.522127] [<ffffffff811e6aec>] io_get_io_group+0x6e/0x94
[ 81.522127] [<ffffffff811e6d8c>] io_group_get_request_list+0x10/0x21
[ 81.522127] [<ffffffff811d7021>] blk_get_request_list+0x9/0xb
[ 81.522127] [<ffffffff811d7ab0>] get_request_wait+0x132/0x17b
[ 81.522127] [<ffffffff811d7dc1>] __make_request+0x2c8/0x396
[ 81.522127] [<ffffffff811d6806>] generic_make_request+0x1f2/0x28c
[ 81.522127] [<ffffffff810e9ee7>] ? bio_init+0x18/0x32
[ 81.522127] [<ffffffff811d8019>] submit_bio+0xb1/0xbc
[ 81.522127] [<ffffffff810e61c1>] submit_bh+0xfb/0x11e
[ 81.522127] [<ffffffff8111f554>] __ext3_get_inode_loc+0x263/0x2c2
[ 81.522127] [<ffffffff81122286>] ext3_iget+0x69/0x399
[ 81.522127] [<ffffffff81125b92>] ext3_lookup+0x81/0xd0
[ 81.522127] [<ffffffff810cfd81>] do_lookup+0xd7/0x15f
[ 81.522127] [<ffffffff810d15c2>] __link_path_walk+0x319/0x67f
[ 81.522127] [<ffffffff810d1976>] path_walk+0x4e/0x97
[ 81.522127] [<ffffffff810d1b48>] do_path_lookup+0x115/0x15a
[ 81.522127] [<ffffffff810d0fec>] ? getname+0x19d/0x1bf
[ 81.522127] [<ffffffff810d252a>] user_path_at+0x52/0x8c
[ 81.522127] [<ffffffff811ee668>] ? __up_read+0x1c/0x8c
[ 81.522127] [<ffffffff8150379b>] ? _spin_unlock_irqrestore+0x3f/0x47
[ 81.522127] [<ffffffff8105f13e>] ? trace_hardirqs_on_caller+0x114/0x138
[ 81.522127] [<ffffffff810cb6c1>] vfs_fstatat+0x35/0x62
[ 81.522127] [<ffffffff811ee6d0>] ? __up_read+0x84/0x8c
[ 81.522127] [<ffffffff810cb7bb>] vfs_stat+0x16/0x18
[ 81.522127] [<ffffffff810cb7d7>] sys_newstat+0x1a/0x34
[ 81.522127] [<ffffffff8100c5e9>] ? retint_swapgs+0xe/0x13
[ 81.522127] [<ffffffff8105f13e>] ? trace_hardirqs_on_caller+0x114/0x138
[ 81.522127] [<ffffffff8107f771>] ? audit_syscall_entry+0xfe/0x12a
[ 81.522127] [<ffffffff8100bb2b>] system_call_fastpath+0x16/0x1bThanks
Vivek
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to a message not in the archiveQuoted 27 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Configure weight=500 ioprio_class=1 on
/dev/hdain this cgroup# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2
Quoted 6 lines
Remove the policy for
/dev/hdain this cgroup# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2
Hi Gui,
Noticed few things during testing.
1. Writing 0 as weight is not removing the policy for me, if I swich the IO scheduler on the device.
- echo "/dev/sdb:500:2" > io.policy
- Change elevator on device /sdb
- echo "/dev/sdb:0:2" > io.policy
- cat io.policy
The old rule is not gone away.
2. One can add same rule twice after chaning elevator.
- echo "/dev/sdb:500:2" > io.policy
- Change elevator on device /sdb
- echo "/dev/sdb:500:2" > io.policy
- cat io.policy
Same rule appears twice
3. If one writes to io.weight, it should not update the weight for a device if there is a rule for the device already. For example, if a cgroup got io.weight=1000 and later i set the weight on /dev/sdb to 500 and then change the io.weight=200, it should not be udpated for for groups on /dev/sdb. Why?, because I think it will make more sense to keep the simple rule that as long there is a rule for device, it always overrides the generic settings of io.weight.
4. Wrong rule should return invalid value instead we see oops.
- echo "/dev/sdb:0:" > io.policy
[ 2651.587533] BUG: unable to handle kernel NULL pointer dereference at (null) [ 2651.588301] IP: [<ffffffff811f035c>] strict_strtoul+0x24/0x79 [ 2651.588301] PGD 38c33067 PUD 38d67067 PMD 0 [ 2651.588301] Oops: 0000 [#2] SMP [ 2651.588301] last sysfs file: /sys/devices/pci0000:00/0000:00:1c.0/0000:0e:00.0/irq [ 2651.588301] CPU 2 [ 2651.588301] Modules linked in: [ 2651.588301] Pid: 4538, comm: bash Tainted: G D 2.6.30-rc4-ioc #52 HP xw6600 Workstation [ 2651.588301] RIP: 0010:[<ffffffff811f035c>] [<ffffffff811f035c>] strict_strtoul+0x24/0x79 [ 2651.588301] RSP: 0018:ffff88003dd73dc0 EFLAGS: 00010286 [ 2651.588301] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffffffffffff [ 2651.588301] RDX: ffff88003e9ffca0 RSI: 000000000000000a RDI: 0000000000000000 [ 2651.588301] RBP: ffff88003dd73de8 R08: 000000000000000a R09: ffff88003dd73cf8 [ 2651.588301] R10: ffff88003dcd2300 R11: ffffffff8178aa00 R12: ffff88003f4a1e00 [ 2651.588301] R13: ffff88003e9ffca0 R14: ffff88003ac5f200 R15: ffff88003fa7ed40 [ 2651.588301] FS: 00007ff971c466f0(0000) GS:ffff88000209c000(0000) knlGS:0000000000000000 [ 2651.588301] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2651.588301] CR2: 0000000000000000 CR3: 000000003ad0d000 CR4: 00000000000006e0 [ 2651.588301] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2651.588301] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2651.588301] Process bash (pid: 4538, threadinfo ffff88003dd72000, task ffff880038d98000) [ 2651.588301] Stack: [ 2651.588301] ffffffff810d8f23 ffff88003fa7ed4a ffff88003dcdeee0 ffff88003f4a1e00 [ 2651.588301] ffff88003e9ffc60 ffff88003dd73e68 ffffffff811e8097 ffff880038dd2780 [ 2651.588301] ffff88003dd73e48 ffff88003fa7ed40 ffff88003fa7ed49 0000000000000000 [ 2651.588301] Call Trace: [ 2651.588301] [<ffffffff810d8f23>] ? iput+0x2f/0x65 [ 2651.588301] [<ffffffff811e8097>] io_cgroup_policy_write+0x11d/0x2ac [ 2651.588301] [<ffffffff81072dee>] cgroup_file_write+0x1ec/0x254 [ 2651.588301] [<ffffffff811afce8>] ? security_file_permission+0x11/0x13 [ 2651.588301] [<ffffffff810c8394>] vfs_write+0xab/0x105 [ 2651.588301] [<ffffffff810c84a8>] sys_write+0x47/0x6c [ 2651.588301] [<ffffffff8100bb2b>] system_call_fastpath+0x16/0x1b [ 2651.588301] Code: 65 ff ff ff 5b c9 c3 55 48 83 c9 ff 31 c0 fc 48 89 e5 41 55 41 89 f0 49 89 d5 41 54 53 48 89 fb 48 83 ec 10 48 c7 02 00 00 00 00 <f2> ae 48 f7 d1 49 89 cc 49 ff cc 74 39 48 8d 75 e0 44 89 c2 48 [ 2651.588301] RIP [<ffffffff811f035c>] strict_strtoul+0x24/0x79 [ 2651.588301] RSP <ffff88003dd73dc0> [ 2651.588301] CR2: 0000000000000000 [ 2651.828110] ---[ end trace 537b9a98ce297f01 ]---
Thanks
Vivek
Quoted 343 lines
Signed-off-by Gui Jianfeng <[email protected]>--- block/elevator-fq.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 11 +++ 2 files changed, 245 insertions(+), 5 deletions(-)block/elevator-fq.c +234
-5@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -1045,12 +1048,30 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +void *key) {struct io_entity *entity = &iog->entity; +struct policy_node *pn; + +spin_lock_irq (&iocg->lock); + pn =policy_search_node (iocg, key); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irq (&iocg->lock); - entity->weight = entity->new_weight = iocg->weight; - entity->ioprio_class = entity->new_ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sched_data = &iog->sched_data; }@@ -1263,7 +1284,7 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,atomic_set (&iog->ref,0 ); iog->deleting =0 ; -io_group_init_entity (iocg, iog); +io_group_init_entity (iocg, iog, key); iog->my_entity = &iog->entity;#ifdef CONFIG_DEBUG_GROUP_IOSCHED iog->iocg_id =css_id (&iocg->css);@@ -1549,8 +1570,208 @@ struct io_group *io_alloc_root_group(struct request_queue *q,return iog; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->list)) + goto out; + +seq_printf (m,"dev weight class\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->list, node) { +seq_printf (m,"%s %lu %lu\n" , pn->dev_name, + pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct policy_node *pn) +{ +list_add (&pn->node, &iocg->list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key) +{ +struct policy_node *pn; + +if (list_empty (&iocg->list)) +return NULL ; + +list_for_each_entry (pn, &iocg->list, node) { +if (pn->key == key) +return pn; + } + +return NULL ; +} + +static void *devname_to_efqd (const char *buf) +{ +struct block_device *bdev; +void *key =NULL ; +struct gendisk *disk; +int part; + + bdev =lookup_bdev (buf); +if (IS_ERR (bdev)) +return NULL ; + + disk =get_gendisk (bdev->bd_dev, &part); + key = (void *)&disk->queue->elevator->efqd; +bdput (bdev); + +return key; +} + +static int policy_parse_and_set (char *buf,struct policy_node *newpn) +{ +char *s[3 ]; +char *p; +int ret; +int i =0 ; + +memset (s,0 ,sizeof (s)); +while (i <ARRAY_SIZE (s)) { + p =strsep (&buf,":" ); +if (!p) +break ; +if (!*p) +continue ; + s[i++] = p; + } + + newpn->key =devname_to_efqd (s[0 ]); +if (!newpn->key) +return -EINVAL ; + +strcpy (newpn->dev_name, s[0 ]); + + ret =strict_strtoul (s[1 ],10 , &newpn->weight); +if (ret || newpn->weight >WEIGHT_MAX ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &newpn->ioprio_class); +if (ret || newpn->ioprio_class <IOPRIO_CLASS_RT || + newpn->ioprio_class >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + +return 0 ; +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->key); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->key == newpn->key) { +if (newpn->weight) { + iog->entity.new_weight = newpn->weight; + iog->entity.new_ioprio_class = + newpn->ioprio_class; +/* +* iog weight and ioprio_class updating +* actually happens if ioprio_changed is set. +* So ensure ioprio_changed is not set until +* new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.new_weight = iocg->weight; + iog->entity.new_ioprio_class = + iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } + } + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +struct cftype bfqio_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,@@ -1592,6 +1813,7 @@ struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_DEFAULT_GRP_WEIGHT ; iocg->ioprio_class =IO_DEFAULT_GRP_CLASS ; +INIT_LIST_HEAD (&iocg->list);return &iocg->css; }@@ -1750,6 +1972,7 @@ void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)unsigned long flags, flags1;int queue_lock_held =0 ;struct elv_fq_data *efqd; +struct policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained @@ -1823,6 +2046,12 @@ locked:BUG_ON (!hlist_empty (&iocg->group_data));free_css_id (&io_subsys, &iocg->css); + +list_for_each_entry_safe (pn, pntmp, &iocg->list, node) { +policy_delete_node (pn); +kfree (pn); + } +kfree (iocg); }@@ -2137,7 +2366,7 @@ void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)void bfq_init_entity (struct io_entity *entity,struct io_group *iog) { entity->ioprio = entity->new_ioprio; - entity->weight = entity->new_weight; + entity->weight = entity->new_weigh; entity->ioprio_class = entity->new_ioprio_class; entity->sched_data = &iog->sched_data; }block/elevator-fq.h +11
-0@@ -253,6 +253,14 @@ struct io_group {#endif }; +struct policy_node { +struct list_head node; +char dev_name[32 ]; +void *key; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup. @@ -269,6 +277,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of policy_node */ +struct list_head list; +spinlock_t lock;struct hlist_head group_data; };Signature · 1 line
1.5.4.rc3
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to Gui JianfengOn Wed, May 13, 2009 at 10:00:21AM +0800, Gui Jianfeng wrote:
[..]
Quoted 14 lines
block/elevator-fq.c +1
-1@@ -2137,7 +2366,7 @@ void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)void bfq_init_entity (struct io_entity *entity,struct io_group *iog) { entity->ioprio = entity->new_ioprio; - entity->weight = entity->new_weight; + entity->weight = entity->new_weigh; entity->ioprio_class = entity->new_ioprio_class; entity->sched_data = &iog->sched_data; }block/elevator-fq.h +0
-0
block/elevator-fq.h struct io_group { · line 256
Quoted 4 lines
block/elevator-fq.h +1
-0@@ -253,6 +253,14 @@ struct io_group {#endif }; +struct policy_node {
Would "io_policy_node" be better?
Quoted 11 lines
block/elevator-fq.h +7
-0+struct list_head node; +char dev_name[32 ]; +void *key; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup.
block/elevator-fq.h struct io_cgroup { · line 282
Quoted 6 lines
block/elevator-fq.h +3
-0@@ -269,6 +277,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of policy_node */ +struct list_head list; +
How about "struct list_head policy_list" or "struct list_head io_policy"?
Thanks
Vivek
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Quoted 20 lines · Vivek Goyal
On Wed, May 13, 2009 at 10:00:21AM +0800, Gui Jianfeng wrote:
[..]
Quoted 17 lines
block/elevator-fq.c +1
-1@@ -2137,7 +2366,7 @@ void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)void bfq_init_entity (struct io_entity *entity,struct io_group *iog) { entity->ioprio = entity->new_ioprio; - entity->weight = entity->new_weight; + entity->weight = entity->new_weigh; entity->ioprio_class = entity->new_ioprio_class; entity->sched_data = &iog->sched_data; }block/elevator-fq.h +0
-0@@ -253,6 +253,14 @@ struct io_group {#endif };
Quoted 3 lines
block/elevator-fq.h struct io_group { · line 256
Quoted 1 line
block/elevator-fq.h +1
-0+struct policy_node {Would "
io_policy_node" be better?
Sure
Quoted 13 lines
Quoted 13 lines
block/elevator-fq.h +7
-0+struct list_head node; +char dev_name[32 ]; +void *key; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup. @@ -269,6 +277,9 @@ struct io_cgroup {
Quoted 7 lines
block/elevator-fq.h struct io_cgroup { · line 282
Quoted 5 lines
block/elevator-fq.h +3
-0unsigned long weight, ioprio_class; +/* list of policy_node */ +struct list_head list; +How about "struct
list_headpolicy_list" or "structlist_headio_policy"?
OK
Signature · 2 lines
Regards Gui Jianfeng
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Quoted 20 lines · Vivek Goyal
On Wed, May 13, 2009 at 10:00:21AM +0800, Gui Jianfeng wrote:
[..]
Quoted 17 lines
block/elevator-fq.c +1
-1@@ -2137,7 +2366,7 @@ void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)void bfq_init_entity (struct io_entity *entity,struct io_group *iog) { entity->ioprio = entity->new_ioprio; - entity->weight = entity->new_weight; + entity->weight = entity->new_weigh; entity->ioprio_class = entity->new_ioprio_class; entity->sched_data = &iog->sched_data; }block/elevator-fq.h +0
-0@@ -253,6 +253,14 @@ struct io_group {#endif };
Quoted 3 lines
block/elevator-fq.h struct io_group { · line 256
Quoted 1 line
block/elevator-fq.h +1
-0+struct policy_node {Would "
io_policy_node" be better?
Sure
Quoted 13 lines
Quoted 13 lines
block/elevator-fq.h +7
-0+struct list_head node; +char dev_name[32 ]; +void *key; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup. @@ -269,6 +277,9 @@ struct io_cgroup {
Quoted 7 lines
block/elevator-fq.h struct io_cgroup { · line 282
Quoted 5 lines
block/elevator-fq.h +3
-0unsigned long weight, ioprio_class; +/* list of policy_node */ +struct list_head list; +How about "struct
list_headpolicy_list" or "structlist_headio_policy"?
OK
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to Gui JianfengQuoted 71 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Configure weight=500 ioprio_class=1 on
/dev/hdain this cgroup# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2Remove the policy for
/dev/hdain this cgroup# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Signed-off-by Gui Jianfeng <[email protected]>--- block/elevator-fq.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 11 +++ 2 files changed, 245 insertions(+), 5 deletions(-)block/elevator-fq.c +14
-1@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -1045,12 +1048,30 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +void *key) {struct io_entity *entity = &iog->entity; +struct policy_node *pn; + +spin_lock_irq (&iocg->lock); + pn =policy_search_node (iocg, key); +if (pn) { + entity->weight = pn->weight;
block/elevator-fq.c struct io_group *io_lookup_io_group_current(struct request_queue *q) · line 1073
Quoted 10 lines
block/elevator-fq.c +10
-0+ entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irq (&iocg->lock);
Hi Gui,
It might make sense to also store the device name or device major and minor number in io_group while creating the io group. This will help us to display io.disk_time and io.disk_sector statistics per device instead of aggregate.
I am attaching a patch I was playing around with to display per device statistics instead of aggregate one. So if user has specified the per device rule.
Thanks
Vivek
o Currently the statistics exported through cgroup are aggregate of statistics on all devices for that cgroup. Instead of aggregate, make these per device.
o Also export another statistics io.disk_dequeue. This keeps a count of how many times a particular group got out of race for the disk. This is a debugging aid to keep a track how often we could create continuously backlogged queues.
--- block/elevator-fq.c | 127 +++++++++++++++++++++++++++++++++------------------- block/elevator-fq.h | 3 + 2 files changed, 85 insertions(+), 45 deletions(-)
block/elevator-fq.h +3 -0
@@ -250,6 +250,9 @@ struct io_group {#ifdef CONFIG_DEBUG_GROUP_IOSCHED unsigned short iocg_id; +dev_t dev; +/* How many times this group has been removed from active tree */ +unsigned long dequeue;#endif };
block/elevator-fq.c +82 -45
232 lines not on this page · raw message
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Vivek Goyal wrote:
...
Quoted 12 lines
Hi Gui,
It might make sense to also store the device name or device major and minor number in
io_groupwhile creating the io group. This will help us to display io.disk_time and io.disk_sector statistics per device instead of aggregate.I am attaching a patch I was playing around with to display per device statistics instead of aggregate one. So if user has specified the per device rule.
Quoted 6 lines
Thanks
Viveko Currently the statistics exported through cgroup are aggregate of statistics on all devices for that cgroup. Instead of aggregate, make these per device.
Hi Vivek,
Actually, I did it also.
FYI
Examples:
# cat io.disk_time dev:/dev/hdb time:4421 dev:others time:3741
# cat io.disk_sectors dev:/dev/hdb sectors:585696 dev:others sectors:2664
--- block/elevator-fq.c | 104 +++++++++++++++++++++++--------------------------- 1 files changed, 48 insertions(+), 56 deletions(-)
block/elevator-fq.c +48 -56
@@ -1162,90 +1162,82 @@ STORE_FUNCTION(weight, 0, WEIGHT_MAX);STORE_FUNCTION (ioprio_class,IOPRIO_CLASS_RT ,IOPRIO_CLASS_IDLE );#undef STORE_FUNCTION -/* -* traverse through all the io_groups associated with this cgroup and calculate -* the aggr disk time received by all the groups on respective disks. -*/ -static u64 calculate_aggr_disk_time (struct io_cgroup *iocg) +static int io_cgroup_disk_time_read (struct cgroup *cgroup, +struct cftype *cftype, +struct seq_file *m) { +struct io_cgroup *iocg;struct io_group *iog;struct hlist_node *n; -u64 disk_time =0 ; - -rcu_read_lock (); -hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) { -/* -* There might be groups which are not functional and -* waiting to be reclaimed upon cgoup deletion. -*/ -if (rcu_dereference (iog->key)) - disk_time += iog->entity.total_service; - } -rcu_read_unlock (); - -return disk_time; -} +struct policy_node *pn; +unsigned int other, time; -static u64 io_cgroup_disk_time_read (struct cgroup *cgroup, -struct cftype *cftype) -{ -struct io_cgroup *iocg; -u64 ret; + other =0 ;if (!cgroup_lock_live_group (cgroup))return -ENODEV ; iocg =cgroup_to_io_cgroup (cgroup);spin_lock_irq (&iocg->lock); - ret =jiffies_to_msecs (calculate_aggr_disk_time (iocg)); +hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) { +if (iog->key !=NULL ) { + pn =policy_search_node (iocg, iog->key); +if (pn) { + time =jiffies_to_msecs (iog->entity. + total_service); +seq_printf (m,"dev:%s time:%u\n" , + pn->dev_name, time); + }else { + other +=jiffies_to_msecs (iog->entity. + total_service); + } + } + } +seq_printf (m,"dev:others time:%u\n" , other); +spin_unlock_irq (&iocg->lock);cgroup_unlock (); -return ret; +return 0 ; } -/* -* traverse through all the io_groups associated with this cgroup and calculate -* the aggr number of sectors transferred by all the groups on respective disks. -*/ -static u64 calculate_aggr_disk_sectors (struct io_cgroup *iocg) +static int io_cgroup_disk_sectors_read (struct cgroup *cgroup, +struct cftype *cftype, +struct seq_file *m) { +struct io_cgroup *iocg;struct io_group *iog;struct hlist_node *n; -u64 disk_sectors =0 ; - -rcu_read_lock (); -hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) { -/* -* There might be groups which are not functional and -* waiting to be reclaimed upon cgoup deletion. -*/ -if (rcu_dereference (iog->key)) - disk_sectors += iog->entity.total_sector_service; - } -rcu_read_unlock (); +struct policy_node *pn; +u64 other =0 ; -return disk_sectors; -} - -static u64 io_cgroup_disk_sectors_read (struct cgroup *cgroup, -struct cftype *cftype) -{ -struct io_cgroup *iocg; -u64 ret;if (!cgroup_lock_live_group (cgroup))return -ENODEV ; iocg =cgroup_to_io_cgroup (cgroup);spin_lock_irq (&iocg->lock); - ret =calculate_aggr_disk_sectors (iocg); +hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) { +if (iog->key) { + pn =policy_search_node (iocg, iog->key); +if (pn) { +seq_printf (m,"dev:%s sectors:%lu\n" , + pn->dev_name, + iog->entity.total_sector_service); + }else { + other += iog->entity.total_sector_service; + } + } + } + +seq_printf (m,"dev:others sectors:%llu\n" , other); +spin_unlock_irq (&iocg->lock);cgroup_unlock (); -return ret; +return 0 ; }/**
@@ -1783,11 +1775,11 @@ struct cftype bfqio_files[] = { }, { .name = "disk_time", - .read_u64 = io_cgroup_disk_time_read, + .read_seq_string = io_cgroup_disk_time_read, }, { .name = "disk_sectors", - .read_u64 = io_cgroup_disk_sectors_read, + .read_seq_string = io_cgroup_disk_sectors_read, }, };
Signature · 1 line
1.5.4.rc3
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Vivek Goyal wrote:
...
Quoted 12 lines
Hi Gui,
It might make sense to also store the device name or device major and minor number in
io_groupwhile creating the io group. This will help us to display io.disk_time and io.disk_sector statistics per device instead of aggregate.I am attaching a patch I was playing around with to display per device statistics instead of aggregate one. So if user has specified the per device rule.
Quoted 6 lines
Thanks
Viveko Currently the statistics exported through cgroup are aggregate of statistics on all devices for that cgroup. Instead of aggregate, make these per device.
Hi Vivek,
Actually, I did it also.
FYI
Examples:
# cat io.disk_time dev:/dev/hdb time:4421 dev:others time:3741
# cat io.disk_sectors dev:/dev/hdb sectors:585696 dev:others sectors:2664
--- block/elevator-fq.c | 104 +++++++++++++++++++++++--------------------------- 1 files changed, 48 insertions(+), 56 deletions(-)
block/elevator-fq.c +48 -56
@@ -1162,90 +1162,82 @@ STORE_FUNCTION(weight, 0, WEIGHT_MAX);STORE_FUNCTION (ioprio_class,IOPRIO_CLASS_RT ,IOPRIO_CLASS_IDLE );#undef STORE_FUNCTION -/* -* traverse through all the io_groups associated with this cgroup and calculate -* the aggr disk time received by all the groups on respective disks. -*/ -static u64 calculate_aggr_disk_time (struct io_cgroup *iocg) +static int io_cgroup_disk_time_read (struct cgroup *cgroup, +struct cftype *cftype, +struct seq_file *m) { +struct io_cgroup *iocg;struct io_group *iog;struct hlist_node *n; -u64 disk_time =0 ; - -rcu_read_lock (); -hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) { -/* -* There might be groups which are not functional and -* waiting to be reclaimed upon cgoup deletion. -*/ -if (rcu_dereference (iog->key)) - disk_time += iog->entity.total_service; - } -rcu_read_unlock (); - -return disk_time; -} +struct policy_node *pn; +unsigned int other, time; -static u64 io_cgroup_disk_time_read (struct cgroup *cgroup, -struct cftype *cftype) -{ -struct io_cgroup *iocg; -u64 ret; + other =0 ;if (!cgroup_lock_live_group (cgroup))return -ENODEV ; iocg =cgroup_to_io_cgroup (cgroup);spin_lock_irq (&iocg->lock); - ret =jiffies_to_msecs (calculate_aggr_disk_time (iocg)); +hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) { +if (iog->key !=NULL ) { + pn =policy_search_node (iocg, iog->key); +if (pn) { + time =jiffies_to_msecs (iog->entity. + total_service); +seq_printf (m,"dev:%s time:%u\n" , + pn->dev_name, time); + }else { + other +=jiffies_to_msecs (iog->entity. + total_service); + } + } + } +seq_printf (m,"dev:others time:%u\n" , other); +spin_unlock_irq (&iocg->lock);cgroup_unlock (); -return ret; +return 0 ; } -/* -* traverse through all the io_groups associated with this cgroup and calculate -* the aggr number of sectors transferred by all the groups on respective disks. -*/ -static u64 calculate_aggr_disk_sectors (struct io_cgroup *iocg) +static int io_cgroup_disk_sectors_read (struct cgroup *cgroup, +struct cftype *cftype, +struct seq_file *m) { +struct io_cgroup *iocg;struct io_group *iog;struct hlist_node *n; -u64 disk_sectors =0 ; - -rcu_read_lock (); -hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) { -/* -* There might be groups which are not functional and -* waiting to be reclaimed upon cgoup deletion. -*/ -if (rcu_dereference (iog->key)) - disk_sectors += iog->entity.total_sector_service; - } -rcu_read_unlock (); +struct policy_node *pn; +u64 other =0 ; -return disk_sectors; -} - -static u64 io_cgroup_disk_sectors_read (struct cgroup *cgroup, -struct cftype *cftype) -{ -struct io_cgroup *iocg; -u64 ret;if (!cgroup_lock_live_group (cgroup))return -ENODEV ; iocg =cgroup_to_io_cgroup (cgroup);spin_lock_irq (&iocg->lock); - ret =calculate_aggr_disk_sectors (iocg); +hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) { +if (iog->key) { + pn =policy_search_node (iocg, iog->key); +if (pn) { +seq_printf (m,"dev:%s sectors:%lu\n" , + pn->dev_name, + iog->entity.total_sector_service); + }else { + other += iog->entity.total_sector_service; + } + } + } + +seq_printf (m,"dev:others sectors:%llu\n" , other); +spin_unlock_irq (&iocg->lock);cgroup_unlock (); -return ret; +return 0 ; }/**
@@ -1783,11 +1775,11 @@ struct cftype bfqio_files[] = { }, { .name = "disk_time", - .read_u64 = io_cgroup_disk_time_read, + .read_seq_string = io_cgroup_disk_time_read, }, { .name = "disk_sectors", - .read_u64 = io_cgroup_disk_sectors_read, + .read_seq_string = io_cgroup_disk_sectors_read, }, };
Signature · 1 line
1.5.4.rc3
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to a message not in the archiveVivek Goyal wrote:
...
Quoted 99 lines
Hi Gui,
It might make sense to also store the device name or device major and minor number in
io_groupwhile creating the io group. This will help us to display io.disk_time and io.disk_sector statistics per device instead of aggregate.I am attaching a patch I was playing around with to display per device statistics instead of aggregate one. So if user has specified the per device rule.
Thanks
Viveko Currently the statistics exported through cgroup are aggregate of statistics on all devices for that cgroup. Instead of aggregate, make these per device.
o Also export another statistics io.disk_dequeue. This keeps a count of how many times a particular group got out of race for the disk. This is a debugging aid to keep a track how often we could create continuously backlogged queues.
Signed-off-by Vivek Goyal <[email protected]>--- block/elevator-fq.c | 127 +++++++++++++++++++++++++++++++++------------------- block/elevator-fq.h | 3 + 2 files changed, 85 insertions(+), 45 deletions(-)Index:
linux14/block/elevator-fq.h
===================================================================block/elevator-fq.h +3
-0@@ -250,6 +250,9 @@ struct io_group {#ifdef CONFIG_DEBUG_GROUP_IOSCHED unsigned short iocg_id; +dev_t dev; +/* How many times this group has been removed from active tree */ +unsigned long dequeue;#endif };Index:
linux14/block/elevator-fq.c
===================================================================block/elevator-fq.c +22
-6@@ -12,6 +12,7 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> /* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -758,6 +759,18 @@ int __bfq_deactivate_entity(struct io_enBUG_ON (sd->active_entity == entity);BUG_ON (sd->next_active == entity); +#ifdef CONFIG_DEBUG_GROUP_IOSCHED + { +struct io_group *iog =io_entity_to_iog (entity); +/* +* Keep track of how many times a group has been removed +* from active tree because it did not have any active +* backlogged ioq under it +*/ +if (iog) + iog->dequeue++; + } +#endif return ret; }@@ -1126,90 +1139,103 @@ STORE_FUNCTION(weight, 0, WEIGHT_MAX);STORE_FUNCTION (ioprio_class,IOPRIO_CLASS_RT ,IOPRIO_CLASS_IDLE );#undef STORE_FUNCTION -/* -* traverse through all the io_groups associated with this cgroup and calculate -* the aggr disk time received by all the groups on respective disks. -*/ -static u64 calculate_aggr_disk_time (struct io_cgroup *iocg) +static int io_cgroup_disk_time_read (struct cgroup *cgroup, +struct cftype *cftype,struct seq_file *m) { +struct io_cgroup *iocg;struct io_group *iog;struct hlist_node *n; -u64 disk_time =0 ; + +if (!cgroup_lock_live_group (cgroup)) +return -ENODEV ; + + iocg =cgroup_to_io_cgroup (cgroup);rcu_read_lock (); +spin_lock_irq (&iocg->lock);hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) {
block/elevator-fq.c STORE_FUNCTION(weight, 0, WEIGHT_MAX); · line 1164
Quoted 10 lines
block/elevator-fq.c +4
-2/* * There might be groups which are not functional and * waiting to be reclaimed upon cgoup deletion. */ -if (rcu_dereference (iog->key)) - disk_time += iog->entity.total_service; +if (rcu_dereference (iog->key)) { +seq_printf (m,"%u %u %lu\n" ,MAJOR (iog->dev), +MINOR (iog->dev), + iog->entity.total_service);
Hi Vivek,
I think it's easier for users if device name is also shown here.
Quoted 11 lines
block/elevator-fq.c +5
-1+ } } +spin_unlock_irq (&iocg->lock);rcu_read_unlock (); -return disk_time; +cgroup_unlock (); + +return 0 ; }
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to Vivek GoyalVivek Goyal wrote:
...
Quoted 99 lines
Hi Gui,
It might make sense to also store the device name or device major and minor number in
io_groupwhile creating the io group. This will help us to display io.disk_time and io.disk_sector statistics per device instead of aggregate.I am attaching a patch I was playing around with to display per device statistics instead of aggregate one. So if user has specified the per device rule.
Thanks
Viveko Currently the statistics exported through cgroup are aggregate of statistics on all devices for that cgroup. Instead of aggregate, make these per device.
o Also export another statistics io.disk_dequeue. This keeps a count of how many times a particular group got out of race for the disk. This is a debugging aid to keep a track how often we could create continuously backlogged queues.
Signed-off-by Vivek Goyal <[email protected]>--- block/elevator-fq.c | 127 +++++++++++++++++++++++++++++++++------------------- block/elevator-fq.h | 3 + 2 files changed, 85 insertions(+), 45 deletions(-)Index:
linux14/block/elevator-fq.h
===================================================================block/elevator-fq.h +3
-0@@ -250,6 +250,9 @@ struct io_group {#ifdef CONFIG_DEBUG_GROUP_IOSCHED unsigned short iocg_id; +dev_t dev; +/* How many times this group has been removed from active tree */ +unsigned long dequeue;#endif };Index:
linux14/block/elevator-fq.c
===================================================================block/elevator-fq.c +22
-6@@ -12,6 +12,7 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> /* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -758,6 +759,18 @@ int __bfq_deactivate_entity(struct io_enBUG_ON (sd->active_entity == entity);BUG_ON (sd->next_active == entity); +#ifdef CONFIG_DEBUG_GROUP_IOSCHED + { +struct io_group *iog =io_entity_to_iog (entity); +/* +* Keep track of how many times a group has been removed +* from active tree because it did not have any active +* backlogged ioq under it +*/ +if (iog) + iog->dequeue++; + } +#endif return ret; }@@ -1126,90 +1139,103 @@ STORE_FUNCTION(weight, 0, WEIGHT_MAX);STORE_FUNCTION (ioprio_class,IOPRIO_CLASS_RT ,IOPRIO_CLASS_IDLE );#undef STORE_FUNCTION -/* -* traverse through all the io_groups associated with this cgroup and calculate -* the aggr disk time received by all the groups on respective disks. -*/ -static u64 calculate_aggr_disk_time (struct io_cgroup *iocg) +static int io_cgroup_disk_time_read (struct cgroup *cgroup, +struct cftype *cftype,struct seq_file *m) { +struct io_cgroup *iocg;struct io_group *iog;struct hlist_node *n; -u64 disk_time =0 ; + +if (!cgroup_lock_live_group (cgroup)) +return -ENODEV ; + + iocg =cgroup_to_io_cgroup (cgroup);rcu_read_lock (); +spin_lock_irq (&iocg->lock);hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) {
block/elevator-fq.c STORE_FUNCTION(weight, 0, WEIGHT_MAX); · line 1164
Quoted 10 lines
block/elevator-fq.c +4
-2/* * There might be groups which are not functional and * waiting to be reclaimed upon cgoup deletion. */ -if (rcu_dereference (iog->key)) - disk_time += iog->entity.total_service; +if (rcu_dereference (iog->key)) { +seq_printf (m,"%u %u %lu\n" ,MAJOR (iog->dev), +MINOR (iog->dev), + iog->entity.total_service);
Hi Vivek,
I think it's easier for users if device name is also shown here.
Quoted 11 lines
block/elevator-fq.c +5
-1+ } } +spin_unlock_irq (&iocg->lock);rcu_read_unlock (); -return disk_time; +cgroup_unlock (); + +return 0 ; }
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to Gui JianfengQuoted 72 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Configure weight=500 ioprio_class=1 on
/dev/hdain this cgroup# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2Remove the policy for
/dev/hdain this cgroup# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Signed-off-by Gui Jianfeng <[email protected]>--- block/elevator-fq.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 11 +++ 2 files changed, 245 insertions(+), 5 deletions(-)block/elevator-fq.c +15
-1@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -1045,12 +1048,30 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +void *key) {struct io_entity *entity = &iog->entity; +struct policy_node *pn; + +spin_lock_irq (&iocg->lock); + pn =policy_search_node (iocg, key); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight;
block/elevator-fq.c struct io_group *io_lookup_io_group_current(struct request_queue *q) · line 1074
Quoted 10 lines
block/elevator-fq.c +9
-0+ entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irq (&iocg->lock);
I think we need to use spin_lock_irqsave() and spin_lock_irqrestore() version above because it can be called with request queue lock held and we don't want to enable the interrupts unconditionally here.
I hit following lock validator warning.
[ 81.521242] =================================
[ 81.522127] [ INFO: inconsistent lock state ]
[ 81.522127] 2.6.30-rc4-ioc #47
[ 81.522127] ---------------------------------
[ 81.522127] inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage.
[ 81.522127] io-group-bw-tes/4138 [HC0[0]:SC0[0]:HE1:SE1] takes:
[ 81.522127] (&q->__queue_lock){+.?...}, at: [<ffffffff811d7b2e>] __make_request+0x35/0x396
[ 81.522127] {IN-SOFTIRQ-W} state was registered at:
[ 81.522127] [<ffffffffffffffff>] 0xffffffffffffffff
[ 81.522127] irq event stamp: 1006
[ 81.522127] hardirqs last enabled at (1005): [<ffffffff810c1198>] kmem_cache_alloc+0x9d/0x105
[ 81.522127] hardirqs last disabled at (1006): [<ffffffff8150343f>] _spin_lock_irq+0x12/0x3e
[ 81.522127] softirqs last enabled at (286): [<ffffffff81042039>] __do_softirq+0x17a/0x187
[ 81.522127] softirqs last disabled at (271): [<ffffffff8100ccfc>] call_softirq+0x1c/0x34
[ 81.522127]
[ 81.522127] other info that might help us debug this:
[ 81.522127] 3 locks held by io-group-bw-tes/4138:
[ 81.522127] #0: (&type->i_mutex_dir_key#4){+.+.+.}, at: [<ffffffff810cfd2c>] do_lookup+0x82/0x15f
[ 81.522127] #1: (&q->__queue_lock){+.?...}, at: [<ffffffff811d7b2e>] __make_request+0x35/0x396
[ 81.522127] #2: (rcu_read_lock){.+.+..}, at: [<ffffffff811e55bb>] __rcu_read_lock+0x0/0x30
[ 81.522127]
[ 81.522127] stack backtrace:
[ 81.522127] Pid: 4138, comm: io-group-bw-tes Not tainted 2.6.30-rc4-ioc #47
[ 81.522127] Call Trace:
[ 81.522127] [<ffffffff8105edad>] valid_state+0x17c/0x18f
[ 81.522127] [<ffffffff8105eb8a>] ? check_usage_backwards+0x0/0x52
[ 81.522127] [<ffffffff8105ee9b>] mark_lock+0xdb/0x1ff
[ 81.522127] [<ffffffff8105f00c>] mark_held_locks+0x4d/0x6b
[ 81.522127] [<ffffffff8150331a>] ? _spin_unlock_irq+0x2b/0x31
[ 81.522127] [<ffffffff8105f13e>] trace_hardirqs_on_caller+0x114/0x138
[ 81.522127] [<ffffffff8105f16f>] trace_hardirqs_on+0xd/0xf
[ 81.522127] [<ffffffff8150331a>] _spin_unlock_irq+0x2b/0x31
[ 81.522127] [<ffffffff811e5534>] ? io_group_init_entity+0x2a/0xb1
[ 81.522127] [<ffffffff811e5597>] io_group_init_entity+0x8d/0xb1
[ 81.522127] [<ffffffff811e688e>] ? io_group_chain_alloc+0x49/0x167
[ 81.522127] [<ffffffff811e68fe>] io_group_chain_alloc+0xb9/0x167
[ 81.522127] [<ffffffff811e6a04>] io_find_alloc_group+0x58/0x85
[ 81.522127] [<ffffffff811e6aec>] io_get_io_group+0x6e/0x94
[ 81.522127] [<ffffffff811e6d8c>] io_group_get_request_list+0x10/0x21
[ 81.522127] [<ffffffff811d7021>] blk_get_request_list+0x9/0xb
[ 81.522127] [<ffffffff811d7ab0>] get_request_wait+0x132/0x17b
[ 81.522127] [<ffffffff811d7dc1>] __make_request+0x2c8/0x396
[ 81.522127] [<ffffffff811d6806>] generic_make_request+0x1f2/0x28c
[ 81.522127] [<ffffffff810e9ee7>] ? bio_init+0x18/0x32
[ 81.522127] [<ffffffff811d8019>] submit_bio+0xb1/0xbc
[ 81.522127] [<ffffffff810e61c1>] submit_bh+0xfb/0x11e
[ 81.522127] [<ffffffff8111f554>] __ext3_get_inode_loc+0x263/0x2c2
[ 81.522127] [<ffffffff81122286>] ext3_iget+0x69/0x399
[ 81.522127] [<ffffffff81125b92>] ext3_lookup+0x81/0xd0
[ 81.522127] [<ffffffff810cfd81>] do_lookup+0xd7/0x15f
[ 81.522127] [<ffffffff810d15c2>] __link_path_walk+0x319/0x67f
[ 81.522127] [<ffffffff810d1976>] path_walk+0x4e/0x97
[ 81.522127] [<ffffffff810d1b48>] do_path_lookup+0x115/0x15a
[ 81.522127] [<ffffffff810d0fec>] ? getname+0x19d/0x1bf
[ 81.522127] [<ffffffff810d252a>] user_path_at+0x52/0x8c
[ 81.522127] [<ffffffff811ee668>] ? __up_read+0x1c/0x8c
[ 81.522127] [<ffffffff8150379b>] ? _spin_unlock_irqrestore+0x3f/0x47
[ 81.522127] [<ffffffff8105f13e>] ? trace_hardirqs_on_caller+0x114/0x138
[ 81.522127] [<ffffffff810cb6c1>] vfs_fstatat+0x35/0x62
[ 81.522127] [<ffffffff811ee6d0>] ? __up_read+0x84/0x8c
[ 81.522127] [<ffffffff810cb7bb>] vfs_stat+0x16/0x18
[ 81.522127] [<ffffffff810cb7d7>] sys_newstat+0x1a/0x34
[ 81.522127] [<ffffffff8100c5e9>] ? retint_swapgs+0xe/0x13
[ 81.522127] [<ffffffff8105f13e>] ? trace_hardirqs_on_caller+0x114/0x138
[ 81.522127] [<ffffffff8107f771>] ? audit_syscall_entry+0xfe/0x12a
[ 81.522127] [<ffffffff8100bb2b>] system_call_fastpath+0x16/0x1bThanks
Vivek
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Vivek Goyal wrote:
...
Quoted 7 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ } + spin_unlock_irq(&iocg->lock);I think we need to use
spin_lock_irqsave()andspin_lock_irqrestore()version above because it can be called with request queue lock held and we don't want to enable the interrupts unconditionally here.
Will change.
Quoted 77 lines
I hit following lock validator warning.
[ 81.521242] ================================= [ 81.522127] [ INFO: inconsistent lock state ] [ 81.522127] 2.6.30-rc4-ioc #47 [ 81.522127] --------------------------------- [ 81.522127] inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage. [ 81.522127] io-group-bw-tes/4138 [HC0[0]:SC0[0]:HE1:SE1] takes: [ 81.522127] (&q->__queue_lock){+.?...}, at: [<ffffffff811d7b2e>] __make_request+0x35/0x396 [ 81.522127] {IN-SOFTIRQ-W} state was registered at: [ 81.522127] [<ffffffffffffffff>] 0xffffffffffffffff [ 81.522127] irq event stamp: 1006 [ 81.522127] hardirqs last enabled at (1005): [<ffffffff810c1198>] kmem_cache_alloc+0x9d/0x105 [ 81.522127] hardirqs last disabled at (1006): [<ffffffff8150343f>] _spin_lock_irq+0x12/0x3e [ 81.522127] softirqs last enabled at (286): [<ffffffff81042039>] __do_softirq+0x17a/0x187 [ 81.522127] softirqs last disabled at (271): [<ffffffff8100ccfc>] call_softirq+0x1c/0x34 [ 81.522127] [ 81.522127] other info that might help us debug this: [ 81.522127] 3 locks held by io-group-bw-tes/4138: [ 81.522127] #0: (&type->i_mutex_dir_key#4){+.+.+.}, at: [<ffffffff810cfd2c>] do_lookup+0x82/0x15f [ 81.522127] #1: (&q->__queue_lock){+.?...}, at: [<ffffffff811d7b2e>] __make_request+0x35/0x396 [ 81.522127] #2: (rcu_read_lock){.+.+..}, at: [<ffffffff811e55bb>] __rcu_read_lock+0x0/0x30 [ 81.522127] [ 81.522127] stack backtrace: [ 81.522127] Pid: 4138, comm: io-group-bw-tes Not tainted 2.6.30-rc4-ioc #47 [ 81.522127] Call Trace: [ 81.522127] [<ffffffff8105edad>] valid_state+0x17c/0x18f [ 81.522127] [<ffffffff8105eb8a>] ? check_usage_backwards+0x0/0x52 [ 81.522127] [<ffffffff8105ee9b>] mark_lock+0xdb/0x1ff [ 81.522127] [<ffffffff8105f00c>] mark_held_locks+0x4d/0x6b [ 81.522127] [<ffffffff8150331a>] ? _spin_unlock_irq+0x2b/0x31 [ 81.522127] [<ffffffff8105f13e>] trace_hardirqs_on_caller+0x114/0x138 [ 81.522127] [<ffffffff8105f16f>] trace_hardirqs_on+0xd/0xf [ 81.522127] [<ffffffff8150331a>] _spin_unlock_irq+0x2b/0x31 [ 81.522127] [<ffffffff811e5534>] ? io_group_init_entity+0x2a/0xb1 [ 81.522127] [<ffffffff811e5597>] io_group_init_entity+0x8d/0xb1 [ 81.522127] [<ffffffff811e688e>] ? io_group_chain_alloc+0x49/0x167 [ 81.522127] [<ffffffff811e68fe>] io_group_chain_alloc+0xb9/0x167 [ 81.522127] [<ffffffff811e6a04>] io_find_alloc_group+0x58/0x85 [ 81.522127] [<ffffffff811e6aec>] io_get_io_group+0x6e/0x94 [ 81.522127] [<ffffffff811e6d8c>] io_group_get_request_list+0x10/0x21 [ 81.522127] [<ffffffff811d7021>] blk_get_request_list+0x9/0xb [ 81.522127] [<ffffffff811d7ab0>] get_request_wait+0x132/0x17b [ 81.522127] [<ffffffff811d7dc1>] __make_request+0x2c8/0x396 [ 81.522127] [<ffffffff811d6806>] generic_make_request+0x1f2/0x28c [ 81.522127] [<ffffffff810e9ee7>] ? bio_init+0x18/0x32 [ 81.522127] [<ffffffff811d8019>] submit_bio+0xb1/0xbc [ 81.522127] [<ffffffff810e61c1>] submit_bh+0xfb/0x11e [ 81.522127] [<ffffffff8111f554>] __ext3_get_inode_loc+0x263/0x2c2 [ 81.522127] [<ffffffff81122286>] ext3_iget+0x69/0x399 [ 81.522127] [<ffffffff81125b92>] ext3_lookup+0x81/0xd0 [ 81.522127] [<ffffffff810cfd81>] do_lookup+0xd7/0x15f [ 81.522127] [<ffffffff810d15c2>] __link_path_walk+0x319/0x67f [ 81.522127] [<ffffffff810d1976>] path_walk+0x4e/0x97 [ 81.522127] [<ffffffff810d1b48>] do_path_lookup+0x115/0x15a [ 81.522127] [<ffffffff810d0fec>] ? getname+0x19d/0x1bf [ 81.522127] [<ffffffff810d252a>] user_path_at+0x52/0x8c [ 81.522127] [<ffffffff811ee668>] ? __up_read+0x1c/0x8c [ 81.522127] [<ffffffff8150379b>] ? _spin_unlock_irqrestore+0x3f/0x47 [ 81.522127] [<ffffffff8105f13e>] ? trace_hardirqs_on_caller+0x114/0x138 [ 81.522127] [<ffffffff810cb6c1>] vfs_fstatat+0x35/0x62 [ 81.522127] [<ffffffff811ee6d0>] ? __up_read+0x84/0x8c [ 81.522127] [<ffffffff810cb7bb>] vfs_stat+0x16/0x18 [ 81.522127] [<ffffffff810cb7d7>] sys_newstat+0x1a/0x34 [ 81.522127] [<ffffffff8100c5e9>] ? retint_swapgs+0xe/0x13 [ 81.522127] [<ffffffff8105f13e>] ? trace_hardirqs_on_caller+0x114/0x138 [ 81.522127] [<ffffffff8107f771>] ? audit_syscall_entry+0xfe/0x12a [ 81.522127] [<ffffffff8100bb2b>] system_call_fastpath+0x16/0x1bThanks
Vivek
Signature · 2 lines
Regards Gui Jianfeng
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Vivek Goyal wrote:
...
Quoted 7 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ } + spin_unlock_irq(&iocg->lock);I think we need to use
spin_lock_irqsave()andspin_lock_irqrestore()version above because it can be called with request queue lock held and we don't want to enable the interrupts unconditionally here.
Will change.
Quoted 77 lines
I hit following lock validator warning.
[ 81.521242] ================================= [ 81.522127] [ INFO: inconsistent lock state ] [ 81.522127] 2.6.30-rc4-ioc #47 [ 81.522127] --------------------------------- [ 81.522127] inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage. [ 81.522127] io-group-bw-tes/4138 [HC0[0]:SC0[0]:HE1:SE1] takes: [ 81.522127] (&q->__queue_lock){+.?...}, at: [<ffffffff811d7b2e>] __make_request+0x35/0x396 [ 81.522127] {IN-SOFTIRQ-W} state was registered at: [ 81.522127] [<ffffffffffffffff>] 0xffffffffffffffff [ 81.522127] irq event stamp: 1006 [ 81.522127] hardirqs last enabled at (1005): [<ffffffff810c1198>] kmem_cache_alloc+0x9d/0x105 [ 81.522127] hardirqs last disabled at (1006): [<ffffffff8150343f>] _spin_lock_irq+0x12/0x3e [ 81.522127] softirqs last enabled at (286): [<ffffffff81042039>] __do_softirq+0x17a/0x187 [ 81.522127] softirqs last disabled at (271): [<ffffffff8100ccfc>] call_softirq+0x1c/0x34 [ 81.522127] [ 81.522127] other info that might help us debug this: [ 81.522127] 3 locks held by io-group-bw-tes/4138: [ 81.522127] #0: (&type->i_mutex_dir_key#4){+.+.+.}, at: [<ffffffff810cfd2c>] do_lookup+0x82/0x15f [ 81.522127] #1: (&q->__queue_lock){+.?...}, at: [<ffffffff811d7b2e>] __make_request+0x35/0x396 [ 81.522127] #2: (rcu_read_lock){.+.+..}, at: [<ffffffff811e55bb>] __rcu_read_lock+0x0/0x30 [ 81.522127] [ 81.522127] stack backtrace: [ 81.522127] Pid: 4138, comm: io-group-bw-tes Not tainted 2.6.30-rc4-ioc #47 [ 81.522127] Call Trace: [ 81.522127] [<ffffffff8105edad>] valid_state+0x17c/0x18f [ 81.522127] [<ffffffff8105eb8a>] ? check_usage_backwards+0x0/0x52 [ 81.522127] [<ffffffff8105ee9b>] mark_lock+0xdb/0x1ff [ 81.522127] [<ffffffff8105f00c>] mark_held_locks+0x4d/0x6b [ 81.522127] [<ffffffff8150331a>] ? _spin_unlock_irq+0x2b/0x31 [ 81.522127] [<ffffffff8105f13e>] trace_hardirqs_on_caller+0x114/0x138 [ 81.522127] [<ffffffff8105f16f>] trace_hardirqs_on+0xd/0xf [ 81.522127] [<ffffffff8150331a>] _spin_unlock_irq+0x2b/0x31 [ 81.522127] [<ffffffff811e5534>] ? io_group_init_entity+0x2a/0xb1 [ 81.522127] [<ffffffff811e5597>] io_group_init_entity+0x8d/0xb1 [ 81.522127] [<ffffffff811e688e>] ? io_group_chain_alloc+0x49/0x167 [ 81.522127] [<ffffffff811e68fe>] io_group_chain_alloc+0xb9/0x167 [ 81.522127] [<ffffffff811e6a04>] io_find_alloc_group+0x58/0x85 [ 81.522127] [<ffffffff811e6aec>] io_get_io_group+0x6e/0x94 [ 81.522127] [<ffffffff811e6d8c>] io_group_get_request_list+0x10/0x21 [ 81.522127] [<ffffffff811d7021>] blk_get_request_list+0x9/0xb [ 81.522127] [<ffffffff811d7ab0>] get_request_wait+0x132/0x17b [ 81.522127] [<ffffffff811d7dc1>] __make_request+0x2c8/0x396 [ 81.522127] [<ffffffff811d6806>] generic_make_request+0x1f2/0x28c [ 81.522127] [<ffffffff810e9ee7>] ? bio_init+0x18/0x32 [ 81.522127] [<ffffffff811d8019>] submit_bio+0xb1/0xbc [ 81.522127] [<ffffffff810e61c1>] submit_bh+0xfb/0x11e [ 81.522127] [<ffffffff8111f554>] __ext3_get_inode_loc+0x263/0x2c2 [ 81.522127] [<ffffffff81122286>] ext3_iget+0x69/0x399 [ 81.522127] [<ffffffff81125b92>] ext3_lookup+0x81/0xd0 [ 81.522127] [<ffffffff810cfd81>] do_lookup+0xd7/0x15f [ 81.522127] [<ffffffff810d15c2>] __link_path_walk+0x319/0x67f [ 81.522127] [<ffffffff810d1976>] path_walk+0x4e/0x97 [ 81.522127] [<ffffffff810d1b48>] do_path_lookup+0x115/0x15a [ 81.522127] [<ffffffff810d0fec>] ? getname+0x19d/0x1bf [ 81.522127] [<ffffffff810d252a>] user_path_at+0x52/0x8c [ 81.522127] [<ffffffff811ee668>] ? __up_read+0x1c/0x8c [ 81.522127] [<ffffffff8150379b>] ? _spin_unlock_irqrestore+0x3f/0x47 [ 81.522127] [<ffffffff8105f13e>] ? trace_hardirqs_on_caller+0x114/0x138 [ 81.522127] [<ffffffff810cb6c1>] vfs_fstatat+0x35/0x62 [ 81.522127] [<ffffffff811ee6d0>] ? __up_read+0x84/0x8c [ 81.522127] [<ffffffff810cb7bb>] vfs_stat+0x16/0x18 [ 81.522127] [<ffffffff810cb7d7>] sys_newstat+0x1a/0x34 [ 81.522127] [<ffffffff8100c5e9>] ? retint_swapgs+0xe/0x13 [ 81.522127] [<ffffffff8105f13e>] ? trace_hardirqs_on_caller+0x114/0x138 [ 81.522127] [<ffffffff8107f771>] ? audit_syscall_entry+0xfe/0x12a [ 81.522127] [<ffffffff8100bb2b>] system_call_fastpath+0x16/0x1bThanks
Vivek
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to Gui JianfengQuoted 27 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Configure weight=500 ioprio_class=1 on
/dev/hdain this cgroup# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2
Quoted 6 lines
Remove the policy for
/dev/hdain this cgroup# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2
Hi Gui,
Noticed few things during testing.
1. Writing 0 as weight is not removing the policy for me, if I swich the IO scheduler on the device.
- echo "/dev/sdb:500:2" > io.policy
- Change elevator on device /sdb
- echo "/dev/sdb:0:2" > io.policy
- cat io.policy
The old rule is not gone away.
2. One can add same rule twice after chaning elevator.
- echo "/dev/sdb:500:2" > io.policy
- Change elevator on device /sdb
- echo "/dev/sdb:500:2" > io.policy
- cat io.policy
Same rule appears twice
3. If one writes to io.weight, it should not update the weight for a device if there is a rule for the device already. For example, if a cgroup got io.weight=1000 and later i set the weight on /dev/sdb to 500 and then change the io.weight=200, it should not be udpated for for groups on /dev/sdb. Why?, because I think it will make more sense to keep the simple rule that as long there is a rule for device, it always overrides the generic settings of io.weight.
4. Wrong rule should return invalid value instead we see oops.
- echo "/dev/sdb:0:" > io.policy
[ 2651.587533] BUG: unable to handle kernel NULL pointer dereference at (null) [ 2651.588301] IP: [<ffffffff811f035c>] strict_strtoul+0x24/0x79 [ 2651.588301] PGD 38c33067 PUD 38d67067 PMD 0 [ 2651.588301] Oops: 0000 [#2] SMP [ 2651.588301] last sysfs file: /sys/devices/pci0000:00/0000:00:1c.0/0000:0e:00.0/irq [ 2651.588301] CPU 2 [ 2651.588301] Modules linked in: [ 2651.588301] Pid: 4538, comm: bash Tainted: G D 2.6.30-rc4-ioc #52 HP xw6600 Workstation [ 2651.588301] RIP: 0010:[<ffffffff811f035c>] [<ffffffff811f035c>] strict_strtoul+0x24/0x79 [ 2651.588301] RSP: 0018:ffff88003dd73dc0 EFLAGS: 00010286 [ 2651.588301] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffffffffffff [ 2651.588301] RDX: ffff88003e9ffca0 RSI: 000000000000000a RDI: 0000000000000000 [ 2651.588301] RBP: ffff88003dd73de8 R08: 000000000000000a R09: ffff88003dd73cf8 [ 2651.588301] R10: ffff88003dcd2300 R11: ffffffff8178aa00 R12: ffff88003f4a1e00 [ 2651.588301] R13: ffff88003e9ffca0 R14: ffff88003ac5f200 R15: ffff88003fa7ed40 [ 2651.588301] FS: 00007ff971c466f0(0000) GS:ffff88000209c000(0000) knlGS:0000000000000000 [ 2651.588301] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2651.588301] CR2: 0000000000000000 CR3: 000000003ad0d000 CR4: 00000000000006e0 [ 2651.588301] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2651.588301] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2651.588301] Process bash (pid: 4538, threadinfo ffff88003dd72000, task ffff880038d98000) [ 2651.588301] Stack: [ 2651.588301] ffffffff810d8f23 ffff88003fa7ed4a ffff88003dcdeee0 ffff88003f4a1e00 [ 2651.588301] ffff88003e9ffc60 ffff88003dd73e68 ffffffff811e8097 ffff880038dd2780 [ 2651.588301] ffff88003dd73e48 ffff88003fa7ed40 ffff88003fa7ed49 0000000000000000 [ 2651.588301] Call Trace: [ 2651.588301] [<ffffffff810d8f23>] ? iput+0x2f/0x65 [ 2651.588301] [<ffffffff811e8097>] io_cgroup_policy_write+0x11d/0x2ac [ 2651.588301] [<ffffffff81072dee>] cgroup_file_write+0x1ec/0x254 [ 2651.588301] [<ffffffff811afce8>] ? security_file_permission+0x11/0x13 [ 2651.588301] [<ffffffff810c8394>] vfs_write+0xab/0x105 [ 2651.588301] [<ffffffff810c84a8>] sys_write+0x47/0x6c [ 2651.588301] [<ffffffff8100bb2b>] system_call_fastpath+0x16/0x1b [ 2651.588301] Code: 65 ff ff ff 5b c9 c3 55 48 83 c9 ff 31 c0 fc 48 89 e5 41 55 41 89 f0 49 89 d5 41 54 53 48 89 fb 48 83 ec 10 48 c7 02 00 00 00 00 <f2> ae 48 f7 d1 49 89 cc 49 ff cc 74 39 48 8d 75 e0 44 89 c2 48 [ 2651.588301] RIP [<ffffffff811f035c>] strict_strtoul+0x24/0x79 [ 2651.588301] RSP <ffff88003dd73dc0> [ 2651.588301] CR2: 0000000000000000 [ 2651.828110] ---[ end trace 537b9a98ce297f01 ]---
Thanks
Vivek
Quoted 343 lines
Signed-off-by Gui Jianfeng <[email protected]>--- block/elevator-fq.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 11 +++ 2 files changed, 245 insertions(+), 5 deletions(-)block/elevator-fq.c +234
-5@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -1045,12 +1048,30 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +void *key) {struct io_entity *entity = &iog->entity; +struct policy_node *pn; + +spin_lock_irq (&iocg->lock); + pn =policy_search_node (iocg, key); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irq (&iocg->lock); - entity->weight = entity->new_weight = iocg->weight; - entity->ioprio_class = entity->new_ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sched_data = &iog->sched_data; }@@ -1263,7 +1284,7 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,atomic_set (&iog->ref,0 ); iog->deleting =0 ; -io_group_init_entity (iocg, iog); +io_group_init_entity (iocg, iog, key); iog->my_entity = &iog->entity;#ifdef CONFIG_DEBUG_GROUP_IOSCHED iog->iocg_id =css_id (&iocg->css);@@ -1549,8 +1570,208 @@ struct io_group *io_alloc_root_group(struct request_queue *q,return iog; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->list)) + goto out; + +seq_printf (m,"dev weight class\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->list, node) { +seq_printf (m,"%s %lu %lu\n" , pn->dev_name, + pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct policy_node *pn) +{ +list_add (&pn->node, &iocg->list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key) +{ +struct policy_node *pn; + +if (list_empty (&iocg->list)) +return NULL ; + +list_for_each_entry (pn, &iocg->list, node) { +if (pn->key == key) +return pn; + } + +return NULL ; +} + +static void *devname_to_efqd (const char *buf) +{ +struct block_device *bdev; +void *key =NULL ; +struct gendisk *disk; +int part; + + bdev =lookup_bdev (buf); +if (IS_ERR (bdev)) +return NULL ; + + disk =get_gendisk (bdev->bd_dev, &part); + key = (void *)&disk->queue->elevator->efqd; +bdput (bdev); + +return key; +} + +static int policy_parse_and_set (char *buf,struct policy_node *newpn) +{ +char *s[3 ]; +char *p; +int ret; +int i =0 ; + +memset (s,0 ,sizeof (s)); +while (i <ARRAY_SIZE (s)) { + p =strsep (&buf,":" ); +if (!p) +break ; +if (!*p) +continue ; + s[i++] = p; + } + + newpn->key =devname_to_efqd (s[0 ]); +if (!newpn->key) +return -EINVAL ; + +strcpy (newpn->dev_name, s[0 ]); + + ret =strict_strtoul (s[1 ],10 , &newpn->weight); +if (ret || newpn->weight >WEIGHT_MAX ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &newpn->ioprio_class); +if (ret || newpn->ioprio_class <IOPRIO_CLASS_RT || + newpn->ioprio_class >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + +return 0 ; +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->key); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->key == newpn->key) { +if (newpn->weight) { + iog->entity.new_weight = newpn->weight; + iog->entity.new_ioprio_class = + newpn->ioprio_class; +/* +* iog weight and ioprio_class updating +* actually happens if ioprio_changed is set. +* So ensure ioprio_changed is not set until +* new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.new_weight = iocg->weight; + iog->entity.new_ioprio_class = + iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } + } + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +struct cftype bfqio_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,@@ -1592,6 +1813,7 @@ struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_DEFAULT_GRP_WEIGHT ; iocg->ioprio_class =IO_DEFAULT_GRP_CLASS ; +INIT_LIST_HEAD (&iocg->list);return &iocg->css; }@@ -1750,6 +1972,7 @@ void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)unsigned long flags, flags1;int queue_lock_held =0 ;struct elv_fq_data *efqd; +struct policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained @@ -1823,6 +2046,12 @@ locked:BUG_ON (!hlist_empty (&iocg->group_data));free_css_id (&io_subsys, &iocg->css); + +list_for_each_entry_safe (pn, pntmp, &iocg->list, node) { +policy_delete_node (pn); +kfree (pn); + } +kfree (iocg); }@@ -2137,7 +2366,7 @@ void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)void bfq_init_entity (struct io_entity *entity,struct io_group *iog) { entity->ioprio = entity->new_ioprio; - entity->weight = entity->new_weight; + entity->weight = entity->new_weigh; entity->ioprio_class = entity->new_ioprio_class; entity->sched_data = &iog->sched_data; }block/elevator-fq.h +11
-0@@ -253,6 +253,14 @@ struct io_group {#endif }; +struct policy_node { +struct list_head node; +char dev_name[32 ]; +void *key; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup. @@ -269,6 +277,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of policy_node */ +struct list_head list; +spinlock_t lock;struct hlist_head group_data; };Signature · 1 line
1.5.4.rc3
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Quoted 65 lines · Vivek Goyal
Quoted 27 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Configure weight=500 ioprio_class=1 on
/dev/hdain this cgroup# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2Quoted 6 lines
Remove the policy for
/dev/hdain this cgroup# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Hi Gui,
Noticed few things during testing.
1. Writing 0 as weight is not removing the policy for me, if I swich the IO scheduler on the device.
- echo "
/dev/sdb:500:2" > io.policy
- Change elevator on device /sdb
- echo "/dev/sdb:0:2" > io.policy
- cat io.policy
The old rule is not gone away.2. One can add same rule twice after chaning elevator.
- echo "
/dev/sdb:500:2" > io.policy
- Change elevator on device /sdb
- echo "/dev/sdb:500:2" > io.policy
- cat io.policySame rule appears twice
3. If one writes to io.weight, it should not update the weight for a device if there is a rule for the device already. For example, if a cgroup got io.weight=1000 and later i set the weight on
/dev/sdbto 500 and then change the io.weight=200, it should not be udpated for for groups on/dev/sdb. Why?, because I think it will make more sense to keep the simple rule that as long there is a rule for device, it always overrides the generic settings of io.weight.
Quoted 3 lines
4. Wrong rule should return invalid value instead we see oops.
- echo "
/dev/sdb:0:" > io.policy
Hi Vivek,
Thanks for testing, i'll fix the above problems, and send an update version.
Quoted 408 lines
[ 2651.587533] BUG: unable to handle kernel NULL pointer dereference at (null) [ 2651.588301] IP: [<ffffffff811f035c>] strict_strtoul+0x24/0x79 [ 2651.588301] PGD 38c33067 PUD 38d67067 PMD 0 [ 2651.588301] Oops: 0000 [#2] SMP [ 2651.588301] last sysfs file: /sys/devices/pci0000:00/0000:00:1c.0/0000:0e:00.0/irq [ 2651.588301] CPU 2 [ 2651.588301] Modules linked in: [ 2651.588301] Pid: 4538, comm: bash Tainted: G D 2.6.30-rc4-ioc #52 HP xw6600 Workstation [ 2651.588301] RIP: 0010:[<ffffffff811f035c>] [<ffffffff811f035c>] strict_strtoul+0x24/0x79 [ 2651.588301] RSP: 0018:ffff88003dd73dc0 EFLAGS: 00010286 [ 2651.588301] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffffffffffff [ 2651.588301] RDX: ffff88003e9ffca0 RSI: 000000000000000a RDI: 0000000000000000 [ 2651.588301] RBP: ffff88003dd73de8 R08: 000000000000000a R09: ffff88003dd73cf8 [ 2651.588301] R10: ffff88003dcd2300 R11: ffffffff8178aa00 R12: ffff88003f4a1e00 [ 2651.588301] R13: ffff88003e9ffca0 R14: ffff88003ac5f200 R15: ffff88003fa7ed40 [ 2651.588301] FS: 00007ff971c466f0(0000) GS:ffff88000209c000(0000) knlGS:0000000000000000 [ 2651.588301] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2651.588301] CR2: 0000000000000000 CR3: 000000003ad0d000 CR4: 00000000000006e0 [ 2651.588301] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2651.588301] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2651.588301] Process bash (pid: 4538, threadinfo ffff88003dd72000, task ffff880038d98000) [ 2651.588301] Stack: [ 2651.588301] ffffffff810d8f23 ffff88003fa7ed4a ffff88003dcdeee0 ffff88003f4a1e00 [ 2651.588301] ffff88003e9ffc60 ffff88003dd73e68 ffffffff811e8097 ffff880038dd2780 [ 2651.588301] ffff88003dd73e48 ffff88003fa7ed40 ffff88003fa7ed49 0000000000000000 [ 2651.588301] Call Trace: [ 2651.588301] [<ffffffff810d8f23>] ? iput+0x2f/0x65 [ 2651.588301] [<ffffffff811e8097>] io_cgroup_policy_write+0x11d/0x2ac [ 2651.588301] [<ffffffff81072dee>] cgroup_file_write+0x1ec/0x254 [ 2651.588301] [<ffffffff811afce8>] ? security_file_permission+0x11/0x13 [ 2651.588301] [<ffffffff810c8394>] vfs_write+0xab/0x105 [ 2651.588301] [<ffffffff810c84a8>] sys_write+0x47/0x6c [ 2651.588301] [<ffffffff8100bb2b>] system_call_fastpath+0x16/0x1b [ 2651.588301] Code: 65 ff ff ff 5b c9 c3 55 48 83 c9 ff 31 c0 fc 48 89 e5 41 55 41 89 f0 49 89 d5 41 54 53 48 89 fb 48 83 ec 10 48 c7 02 00 00 00 00 <f2> ae 48 f7 d1 49 89 cc 49 ff cc 74 39 48 8d 75 e0 44 89 c2 48 [ 2651.588301] RIP [<ffffffff811f035c>] strict_strtoul+0x24/0x79 [ 2651.588301] RSP <ffff88003dd73dc0> [ 2651.588301] CR2: 0000000000000000 [ 2651.828110] ---[ end trace 537b9a98ce297f01 ]---Thanks
VivekQuoted 343 lines
Signed-off-by Gui Jianfeng <[email protected]>--- block/elevator-fq.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 11 +++ 2 files changed, 245 insertions(+), 5 deletions(-)block/elevator-fq.c +234
-5@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -1045,12 +1048,30 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +void *key) {struct io_entity *entity = &iog->entity; +struct policy_node *pn; + +spin_lock_irq (&iocg->lock); + pn =policy_search_node (iocg, key); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irq (&iocg->lock); - entity->weight = entity->new_weight = iocg->weight; - entity->ioprio_class = entity->new_ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sched_data = &iog->sched_data; }@@ -1263,7 +1284,7 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,atomic_set (&iog->ref,0 ); iog->deleting =0 ; -io_group_init_entity (iocg, iog); +io_group_init_entity (iocg, iog, key); iog->my_entity = &iog->entity;#ifdef CONFIG_DEBUG_GROUP_IOSCHED iog->iocg_id =css_id (&iocg->css);@@ -1549,8 +1570,208 @@ struct io_group *io_alloc_root_group(struct request_queue *q,return iog; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->list)) + goto out; + +seq_printf (m,"dev weight class\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->list, node) { +seq_printf (m,"%s %lu %lu\n" , pn->dev_name, + pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct policy_node *pn) +{ +list_add (&pn->node, &iocg->list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key) +{ +struct policy_node *pn; + +if (list_empty (&iocg->list)) +return NULL ; + +list_for_each_entry (pn, &iocg->list, node) { +if (pn->key == key) +return pn; + } + +return NULL ; +} + +static void *devname_to_efqd (const char *buf) +{ +struct block_device *bdev; +void *key =NULL ; +struct gendisk *disk; +int part; + + bdev =lookup_bdev (buf); +if (IS_ERR (bdev)) +return NULL ; + + disk =get_gendisk (bdev->bd_dev, &part); + key = (void *)&disk->queue->elevator->efqd; +bdput (bdev); + +return key; +} + +static int policy_parse_and_set (char *buf,struct policy_node *newpn) +{ +char *s[3 ]; +char *p; +int ret; +int i =0 ; + +memset (s,0 ,sizeof (s)); +while (i <ARRAY_SIZE (s)) { + p =strsep (&buf,":" ); +if (!p) +break ; +if (!*p) +continue ; + s[i++] = p; + } + + newpn->key =devname_to_efqd (s[0 ]); +if (!newpn->key) +return -EINVAL ; + +strcpy (newpn->dev_name, s[0 ]); + + ret =strict_strtoul (s[1 ],10 , &newpn->weight); +if (ret || newpn->weight >WEIGHT_MAX ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &newpn->ioprio_class); +if (ret || newpn->ioprio_class <IOPRIO_CLASS_RT || + newpn->ioprio_class >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + +return 0 ; +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->key); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->key == newpn->key) { +if (newpn->weight) { + iog->entity.new_weight = newpn->weight; + iog->entity.new_ioprio_class = + newpn->ioprio_class; +/* +* iog weight and ioprio_class updating +* actually happens if ioprio_changed is set. +* So ensure ioprio_changed is not set until +* new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.new_weight = iocg->weight; + iog->entity.new_ioprio_class = + iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } + } + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +struct cftype bfqio_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,@@ -1592,6 +1813,7 @@ struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_DEFAULT_GRP_WEIGHT ; iocg->ioprio_class =IO_DEFAULT_GRP_CLASS ; +INIT_LIST_HEAD (&iocg->list);return &iocg->css; }@@ -1750,6 +1972,7 @@ void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)unsigned long flags, flags1;int queue_lock_held =0 ;struct elv_fq_data *efqd; +struct policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained @@ -1823,6 +2046,12 @@ locked:BUG_ON (!hlist_empty (&iocg->group_data));free_css_id (&io_subsys, &iocg->css); + +list_for_each_entry_safe (pn, pntmp, &iocg->list, node) { +policy_delete_node (pn); +kfree (pn); + } +kfree (iocg); }@@ -2137,7 +2366,7 @@ void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)void bfq_init_entity (struct io_entity *entity,struct io_group *iog) { entity->ioprio = entity->new_ioprio; - entity->weight = entity->new_weight; + entity->weight = entity->new_weigh; entity->ioprio_class = entity->new_ioprio_class; entity->sched_data = &iog->sched_data; }block/elevator-fq.h +11
-0@@ -253,6 +253,14 @@ struct io_group {#endif }; +struct policy_node { +struct list_head node; +char dev_name[32 ]; +void *key; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup. @@ -269,6 +277,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of policy_node */ +struct list_head list; +spinlock_t lock;struct hlist_head group_data; };Signature · 1 line
1.5.4.rc3
Signature · 2 lines
Regards Gui Jianfeng
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Quoted 65 lines · Vivek Goyal
Quoted 27 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Configure weight=500 ioprio_class=1 on
/dev/hdain this cgroup# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2Quoted 6 lines
Remove the policy for
/dev/hdain this cgroup# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Hi Gui,
Noticed few things during testing.
1. Writing 0 as weight is not removing the policy for me, if I swich the IO scheduler on the device.
- echo "
/dev/sdb:500:2" > io.policy
- Change elevator on device /sdb
- echo "/dev/sdb:0:2" > io.policy
- cat io.policy
The old rule is not gone away.2. One can add same rule twice after chaning elevator.
- echo "
/dev/sdb:500:2" > io.policy
- Change elevator on device /sdb
- echo "/dev/sdb:500:2" > io.policy
- cat io.policySame rule appears twice
3. If one writes to io.weight, it should not update the weight for a device if there is a rule for the device already. For example, if a cgroup got io.weight=1000 and later i set the weight on
/dev/sdbto 500 and then change the io.weight=200, it should not be udpated for for groups on/dev/sdb. Why?, because I think it will make more sense to keep the simple rule that as long there is a rule for device, it always overrides the generic settings of io.weight.
Quoted 3 lines
4. Wrong rule should return invalid value instead we see oops.
- echo "
/dev/sdb:0:" > io.policy
Hi Vivek,
Thanks for testing, i'll fix the above problems, and send an update version.
Quoted 408 lines
[ 2651.587533] BUG: unable to handle kernel NULL pointer dereference at (null) [ 2651.588301] IP: [<ffffffff811f035c>] strict_strtoul+0x24/0x79 [ 2651.588301] PGD 38c33067 PUD 38d67067 PMD 0 [ 2651.588301] Oops: 0000 [#2] SMP [ 2651.588301] last sysfs file: /sys/devices/pci0000:00/0000:00:1c.0/0000:0e:00.0/irq [ 2651.588301] CPU 2 [ 2651.588301] Modules linked in: [ 2651.588301] Pid: 4538, comm: bash Tainted: G D 2.6.30-rc4-ioc #52 HP xw6600 Workstation [ 2651.588301] RIP: 0010:[<ffffffff811f035c>] [<ffffffff811f035c>] strict_strtoul+0x24/0x79 [ 2651.588301] RSP: 0018:ffff88003dd73dc0 EFLAGS: 00010286 [ 2651.588301] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffffffffffff [ 2651.588301] RDX: ffff88003e9ffca0 RSI: 000000000000000a RDI: 0000000000000000 [ 2651.588301] RBP: ffff88003dd73de8 R08: 000000000000000a R09: ffff88003dd73cf8 [ 2651.588301] R10: ffff88003dcd2300 R11: ffffffff8178aa00 R12: ffff88003f4a1e00 [ 2651.588301] R13: ffff88003e9ffca0 R14: ffff88003ac5f200 R15: ffff88003fa7ed40 [ 2651.588301] FS: 00007ff971c466f0(0000) GS:ffff88000209c000(0000) knlGS:0000000000000000 [ 2651.588301] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 2651.588301] CR2: 0000000000000000 CR3: 000000003ad0d000 CR4: 00000000000006e0 [ 2651.588301] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2651.588301] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 2651.588301] Process bash (pid: 4538, threadinfo ffff88003dd72000, task ffff880038d98000) [ 2651.588301] Stack: [ 2651.588301] ffffffff810d8f23 ffff88003fa7ed4a ffff88003dcdeee0 ffff88003f4a1e00 [ 2651.588301] ffff88003e9ffc60 ffff88003dd73e68 ffffffff811e8097 ffff880038dd2780 [ 2651.588301] ffff88003dd73e48 ffff88003fa7ed40 ffff88003fa7ed49 0000000000000000 [ 2651.588301] Call Trace: [ 2651.588301] [<ffffffff810d8f23>] ? iput+0x2f/0x65 [ 2651.588301] [<ffffffff811e8097>] io_cgroup_policy_write+0x11d/0x2ac [ 2651.588301] [<ffffffff81072dee>] cgroup_file_write+0x1ec/0x254 [ 2651.588301] [<ffffffff811afce8>] ? security_file_permission+0x11/0x13 [ 2651.588301] [<ffffffff810c8394>] vfs_write+0xab/0x105 [ 2651.588301] [<ffffffff810c84a8>] sys_write+0x47/0x6c [ 2651.588301] [<ffffffff8100bb2b>] system_call_fastpath+0x16/0x1b [ 2651.588301] Code: 65 ff ff ff 5b c9 c3 55 48 83 c9 ff 31 c0 fc 48 89 e5 41 55 41 89 f0 49 89 d5 41 54 53 48 89 fb 48 83 ec 10 48 c7 02 00 00 00 00 <f2> ae 48 f7 d1 49 89 cc 49 ff cc 74 39 48 8d 75 e0 44 89 c2 48 [ 2651.588301] RIP [<ffffffff811f035c>] strict_strtoul+0x24/0x79 [ 2651.588301] RSP <ffff88003dd73dc0> [ 2651.588301] CR2: 0000000000000000 [ 2651.828110] ---[ end trace 537b9a98ce297f01 ]---Thanks
VivekQuoted 343 lines
Signed-off-by Gui Jianfeng <[email protected]>--- block/elevator-fq.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 11 +++ 2 files changed, 245 insertions(+), 5 deletions(-)block/elevator-fq.c +234
-5@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -1045,12 +1048,30 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +void *key) {struct io_entity *entity = &iog->entity; +struct policy_node *pn; + +spin_lock_irq (&iocg->lock); + pn =policy_search_node (iocg, key); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irq (&iocg->lock); - entity->weight = entity->new_weight = iocg->weight; - entity->ioprio_class = entity->new_ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sched_data = &iog->sched_data; }@@ -1263,7 +1284,7 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,atomic_set (&iog->ref,0 ); iog->deleting =0 ; -io_group_init_entity (iocg, iog); +io_group_init_entity (iocg, iog, key); iog->my_entity = &iog->entity;#ifdef CONFIG_DEBUG_GROUP_IOSCHED iog->iocg_id =css_id (&iocg->css);@@ -1549,8 +1570,208 @@ struct io_group *io_alloc_root_group(struct request_queue *q,return iog; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->list)) + goto out; + +seq_printf (m,"dev weight class\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->list, node) { +seq_printf (m,"%s %lu %lu\n" , pn->dev_name, + pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct policy_node *pn) +{ +list_add (&pn->node, &iocg->list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct policy_node *policy_search_node (const struct io_cgroup *iocg, +void *key) +{ +struct policy_node *pn; + +if (list_empty (&iocg->list)) +return NULL ; + +list_for_each_entry (pn, &iocg->list, node) { +if (pn->key == key) +return pn; + } + +return NULL ; +} + +static void *devname_to_efqd (const char *buf) +{ +struct block_device *bdev; +void *key =NULL ; +struct gendisk *disk; +int part; + + bdev =lookup_bdev (buf); +if (IS_ERR (bdev)) +return NULL ; + + disk =get_gendisk (bdev->bd_dev, &part); + key = (void *)&disk->queue->elevator->efqd; +bdput (bdev); + +return key; +} + +static int policy_parse_and_set (char *buf,struct policy_node *newpn) +{ +char *s[3 ]; +char *p; +int ret; +int i =0 ; + +memset (s,0 ,sizeof (s)); +while (i <ARRAY_SIZE (s)) { + p =strsep (&buf,":" ); +if (!p) +break ; +if (!*p) +continue ; + s[i++] = p; + } + + newpn->key =devname_to_efqd (s[0 ]); +if (!newpn->key) +return -EINVAL ; + +strcpy (newpn->dev_name, s[0 ]); + + ret =strict_strtoul (s[1 ],10 , &newpn->weight); +if (ret || newpn->weight >WEIGHT_MAX ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &newpn->ioprio_class); +if (ret || newpn->ioprio_class <IOPRIO_CLASS_RT || + newpn->ioprio_class >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + +return 0 ; +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->key); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->key == newpn->key) { +if (newpn->weight) { + iog->entity.new_weight = newpn->weight; + iog->entity.new_ioprio_class = + newpn->ioprio_class; +/* +* iog weight and ioprio_class updating +* actually happens if ioprio_changed is set. +* So ensure ioprio_changed is not set until +* new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.new_weight = iocg->weight; + iog->entity.new_ioprio_class = + iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } + } + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +struct cftype bfqio_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,@@ -1592,6 +1813,7 @@ struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_DEFAULT_GRP_WEIGHT ; iocg->ioprio_class =IO_DEFAULT_GRP_CLASS ; +INIT_LIST_HEAD (&iocg->list);return &iocg->css; }@@ -1750,6 +1972,7 @@ void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)unsigned long flags, flags1;int queue_lock_held =0 ;struct elv_fq_data *efqd; +struct policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained @@ -1823,6 +2046,12 @@ locked:BUG_ON (!hlist_empty (&iocg->group_data));free_css_id (&io_subsys, &iocg->css); + +list_for_each_entry_safe (pn, pntmp, &iocg->list, node) { +policy_delete_node (pn); +kfree (pn); + } +kfree (iocg); }@@ -2137,7 +2366,7 @@ void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)void bfq_init_entity (struct io_entity *entity,struct io_group *iog) { entity->ioprio = entity->new_ioprio; - entity->weight = entity->new_weight; + entity->weight = entity->new_weigh; entity->ioprio_class = entity->new_ioprio_class; entity->sched_data = &iog->sched_data; }block/elevator-fq.h +11
-0@@ -253,6 +253,14 @@ struct io_group {#endif }; +struct policy_node { +struct list_head node; +char dev_name[32 ]; +void *key; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup. @@ -269,6 +277,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of policy_node */ +struct list_head list; +spinlock_t lock;struct hlist_head group_data; };Signature · 1 line
1.5.4.rc3
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to a message not in the archiveHi Vivek,
This patch enables per-cgroup per-device weight and ioprio_class handling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight and ioprio_class for each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.
You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on /dev/hdb in this cgroup
# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2
Configure weight=500 ioprio_class=1 on /dev/hda in this cgroup
# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2
Remove the policy for /dev/hda in this cgroup
# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2
Changelog (v1 -> v2)
- Rename some structures
- Use spin_lock_irqsave() and spin_lock_irqrestore() version to prevent from enabling the interrupts unconditionally.
- Fix policy setup bug when switching to another io scheduler.
- If a policy is available for a specific device, don't update weight and io class when writing "weight" and "iprio_class".
- Fix a bug when parsing policy string.
--- block/elevator-fq.c | 258 +++++++++++++++++++++++++++++++++++++++++++++++++-- block/elevator-fq.h | 12 +++ 2 files changed, 261 insertions(+), 9 deletions(-)
block/elevator-fq.c +249 -9
@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;
@@ -1045,12 +1048,31 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct io_policy_node *policy_search_node (const struct io_cgroup *iocg, +dev_t dev); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +dev_t dev) {struct io_entity *entity = &iog->entity; +struct io_policy_node *pn; +unsigned long flags; + +spin_lock_irqsave (&iocg->lock, flags); + pn =policy_search_node (iocg, dev); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irqrestore (&iocg->lock, flags); - entity->weight = entity->new_weight = iocg->weight; - entity->ioprio_class = entity->new_ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sched_data = &iog->sched_data; }
@@ -1114,6 +1136,7 @@ static int io_cgroup_##__VAR##_write(struct cgroup *cgroup, \struct io_cgroup *iocg; \struct io_group *iog; \struct hlist_node *n; \ +struct io_policy_node *pn; \ \if (val < (__MIN) || val > (__MAX)) \return -EINVAL ; \
@@ -1126,6 +1149,9 @@ static int io_cgroup_##__VAR##_write(struct cgroup *cgroup, \spin_lock_irq (&iocg->lock); \ iocg->__VAR = (unsigned long )val; \ hlist_for_each_entry(iog, n, &iocg->group_data, group_node) { \ + pn =policy_search_node (iocg, iog->dev); \ +if (pn) \ +continue ; \ iog->entity.new_##__VAR = (unsigned long)val; \smp_wmb (); \ iog->entity.ioprio_changed =1 ; \
@@ -1237,7 +1263,7 @@ static u64 io_cgroup_disk_sectors_read(struct cgroup *cgroup,* to the root has already an allocated group on @bfqd. */ struct io_group *io_group_chain_alloc (struct request_queue *q,void *key, -struct cgroup *cgroup) +struct cgroup *cgroup,struct bio *bio) {struct io_cgroup *iocg;struct io_group *iog, *leaf =NULL , *prev =NULL ;
@@ -1263,12 +1289,17 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,atomic_set (&iog->ref,0 ); iog->deleting =0 ; -io_group_init_entity (iocg, iog); - iog->my_entity = &iog->entity;#ifdef CONFIG_DEBUG_GROUP_IOSCHED iog->iocg_id =css_id (&iocg->css); +if (bio) { +struct gendisk *disk = bio->bi_bdev->bd_disk; + iog->dev =MKDEV (disk->major, disk->first_minor); + }#endif +io_group_init_entity (iocg, iog, iog->dev); + iog->my_entity = &iog->entity; +blk_init_request_list (&iog->rl);if (leaf ==NULL ) {
@@ -1379,7 +1410,7 @@ void io_group_chain_link(struct request_queue *q, void *key,*/ struct io_group *io_find_alloc_group (struct request_queue *q,struct cgroup *cgroup,struct elv_fq_data *efqd, -int create) +int create,struct bio *bio) {struct io_cgroup *iocg =cgroup_to_io_cgroup (cgroup);struct io_group *iog =NULL ;
@@ -1390,7 +1421,7 @@ struct io_group *io_find_alloc_group(struct request_queue *q,if (iog !=NULL || !create)return iog; - iog =io_group_chain_alloc (q, key, cgroup); + iog =io_group_chain_alloc (q, key, cgroup, bio);if (iog !=NULL )io_group_chain_link (q, key, cgroup, iog, efqd);
@@ -1489,7 +1520,7 @@ struct io_group *io_get_io_group_bio(struct request_queue *q, struct bio *bio, goto out; } - iog =io_find_alloc_group (q, cgroup, efqd, create); + iog =io_find_alloc_group (q, cgroup, efqd, create, bio);if (!iog) {if (create) iog = efqd->root_group;
@@ -1549,8 +1580,209 @@ struct io_group *io_alloc_root_group(struct request_queue *q,return iog; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct io_policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->policy_list)) + goto out; + +seq_printf (m,"dev weight class\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->policy_list, node) { +seq_printf (m,"%s %lu %lu\n" , pn->dev_name, + pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct io_policy_node *pn) +{ +list_add (&pn->node, &iocg->policy_list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct io_policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct io_policy_node *policy_search_node (const struct io_cgroup *iocg, +dev_t dev) +{ +struct io_policy_node *pn; + +if (list_empty (&iocg->policy_list)) +return NULL ; + +list_for_each_entry (pn, &iocg->policy_list, node) { +if (pn->dev == dev) +return pn; + } + +return NULL ; +} + +static int devname_to_devnum (const char *buf,dev_t *dev) +{ +struct block_device *bdev; +struct gendisk *disk; +int part; + + bdev =lookup_bdev (buf); +if (IS_ERR (bdev)) +return -ENODEV ; + + disk =get_gendisk (bdev->bd_dev, &part); + *dev =MKDEV (disk->major, disk->first_minor); +bdput (bdev); + +return 0 ; +} + +static int policy_parse_and_set (char *buf,struct io_policy_node *newpn) +{ +char *s[3 ], *p; +int ret; +int i =0 ; + +memset (s,0 ,sizeof (s)); +while ((p =strsep (&buf,":" )) !=NULL ) { +if (!*p) +continue ; + s[i++] = p; + } + + ret =devname_to_devnum (s[0 ], &newpn->dev); +if (ret) +return ret; + +strcpy (newpn->dev_name, s[0 ]); + +if (s[1 ] ==NULL ) +return -EINVAL ; + + ret =strict_strtoul (s[1 ],10 , &newpn->weight); +if (ret || newpn->weight >WEIGHT_MAX ) +return -EINVAL ; + +if (s[2 ] ==NULL ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &newpn->ioprio_class); +if (ret || newpn->ioprio_class <IOPRIO_CLASS_RT || + newpn->ioprio_class >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + +return 0 ; +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct io_policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->dev); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->dev == newpn->dev) { +if (newpn->weight) { + iog->entity.new_weight = newpn->weight; + iog->entity.new_ioprio_class = + newpn->ioprio_class; +/* +* iog weight and ioprio_class updating +* actually happens if ioprio_changed is set. +* So ensure ioprio_changed is not set until +* new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.new_weight = iocg->weight; + iog->entity.new_ioprio_class = + iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } + } + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +struct cftype bfqio_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,
@@ -1592,6 +1824,7 @@ struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_DEFAULT_GRP_WEIGHT ; iocg->ioprio_class =IO_DEFAULT_GRP_CLASS ; +INIT_LIST_HEAD (&iocg->policy_list);return &iocg->css; }
@@ -1750,6 +1983,7 @@ void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)unsigned long flags, flags1;int queue_lock_held =0 ;struct elv_fq_data *efqd; +struct io_policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained
@@ -1823,6 +2057,12 @@ locked:BUG_ON (!hlist_empty (&iocg->group_data));free_css_id (&io_subsys, &iocg->css); + +list_for_each_entry_safe (pn, pntmp, &iocg->policy_list, node) { +policy_delete_node (pn); +kfree (pn); + } +kfree (iocg); }
block/elevator-fq.h +12 -0
@@ -250,9 +250,18 @@ struct io_group {#ifdef CONFIG_DEBUG_GROUP_IOSCHED unsigned short iocg_id; +dev_t dev;#endif }; +struct io_policy_node { +struct list_head node; +char dev_name[32 ]; +dev_t dev; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup.
@@ -269,6 +278,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of io_policy_node */ +struct list_head policy_list; +spinlock_t lock;struct hlist_head group_data; };
Signature · 1 line
1.5.4.rc3
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to Vivek GoyalHi Vivek,
This patch enables per-cgroup per-device weight and ioprio_class handling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight and ioprio_class for each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.
You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on /dev/hdb in this cgroup
# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2
Configure weight=500 ioprio_class=1 on /dev/hda in this cgroup
# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2
Remove the policy for /dev/hda in this cgroup
# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2
Changelog (v1 -> v2)
- Rename some structures
- Use spin_lock_irqsave() and spin_lock_irqrestore() version to prevent from enabling the interrupts unconditionally.
- Fix policy setup bug when switching to another io scheduler.
- If a policy is available for a specific device, don't update weight and io class when writing "weight" and "iprio_class".
- Fix a bug when parsing policy string.
--- block/elevator-fq.c | 258 +++++++++++++++++++++++++++++++++++++++++++++++++-- block/elevator-fq.h | 12 +++ 2 files changed, 261 insertions(+), 9 deletions(-)
block/elevator-fq.c +249 -9
@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;
@@ -1045,12 +1048,31 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct io_policy_node *policy_search_node (const struct io_cgroup *iocg, +dev_t dev); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +dev_t dev) {struct io_entity *entity = &iog->entity; +struct io_policy_node *pn; +unsigned long flags; + +spin_lock_irqsave (&iocg->lock, flags); + pn =policy_search_node (iocg, dev); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irqrestore (&iocg->lock, flags); - entity->weight = entity->new_weight = iocg->weight; - entity->ioprio_class = entity->new_ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sched_data = &iog->sched_data; }
@@ -1114,6 +1136,7 @@ static int io_cgroup_##__VAR##_write(struct cgroup *cgroup, \struct io_cgroup *iocg; \struct io_group *iog; \struct hlist_node *n; \ +struct io_policy_node *pn; \ \if (val < (__MIN) || val > (__MAX)) \return -EINVAL ; \
@@ -1126,6 +1149,9 @@ static int io_cgroup_##__VAR##_write(struct cgroup *cgroup, \spin_lock_irq (&iocg->lock); \ iocg->__VAR = (unsigned long )val; \ hlist_for_each_entry(iog, n, &iocg->group_data, group_node) { \ + pn =policy_search_node (iocg, iog->dev); \ +if (pn) \ +continue ; \ iog->entity.new_##__VAR = (unsigned long)val; \smp_wmb (); \ iog->entity.ioprio_changed =1 ; \
@@ -1237,7 +1263,7 @@ static u64 io_cgroup_disk_sectors_read(struct cgroup *cgroup,* to the root has already an allocated group on @bfqd. */ struct io_group *io_group_chain_alloc (struct request_queue *q,void *key, -struct cgroup *cgroup) +struct cgroup *cgroup,struct bio *bio) {struct io_cgroup *iocg;struct io_group *iog, *leaf =NULL , *prev =NULL ;
@@ -1263,12 +1289,17 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,atomic_set (&iog->ref,0 ); iog->deleting =0 ; -io_group_init_entity (iocg, iog); - iog->my_entity = &iog->entity;#ifdef CONFIG_DEBUG_GROUP_IOSCHED iog->iocg_id =css_id (&iocg->css); +if (bio) { +struct gendisk *disk = bio->bi_bdev->bd_disk; + iog->dev =MKDEV (disk->major, disk->first_minor); + }#endif +io_group_init_entity (iocg, iog, iog->dev); + iog->my_entity = &iog->entity; +blk_init_request_list (&iog->rl);if (leaf ==NULL ) {
@@ -1379,7 +1410,7 @@ void io_group_chain_link(struct request_queue *q, void *key,*/ struct io_group *io_find_alloc_group (struct request_queue *q,struct cgroup *cgroup,struct elv_fq_data *efqd, -int create) +int create,struct bio *bio) {struct io_cgroup *iocg =cgroup_to_io_cgroup (cgroup);struct io_group *iog =NULL ;
@@ -1390,7 +1421,7 @@ struct io_group *io_find_alloc_group(struct request_queue *q,if (iog !=NULL || !create)return iog; - iog =io_group_chain_alloc (q, key, cgroup); + iog =io_group_chain_alloc (q, key, cgroup, bio);if (iog !=NULL )io_group_chain_link (q, key, cgroup, iog, efqd);
@@ -1489,7 +1520,7 @@ struct io_group *io_get_io_group_bio(struct request_queue *q, struct bio *bio, goto out; } - iog =io_find_alloc_group (q, cgroup, efqd, create); + iog =io_find_alloc_group (q, cgroup, efqd, create, bio);if (!iog) {if (create) iog = efqd->root_group;
@@ -1549,8 +1580,209 @@ struct io_group *io_alloc_root_group(struct request_queue *q,return iog; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct io_policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->policy_list)) + goto out; + +seq_printf (m,"dev weight class\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->policy_list, node) { +seq_printf (m,"%s %lu %lu\n" , pn->dev_name, + pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct io_policy_node *pn) +{ +list_add (&pn->node, &iocg->policy_list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct io_policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct io_policy_node *policy_search_node (const struct io_cgroup *iocg, +dev_t dev) +{ +struct io_policy_node *pn; + +if (list_empty (&iocg->policy_list)) +return NULL ; + +list_for_each_entry (pn, &iocg->policy_list, node) { +if (pn->dev == dev) +return pn; + } + +return NULL ; +} + +static int devname_to_devnum (const char *buf,dev_t *dev) +{ +struct block_device *bdev; +struct gendisk *disk; +int part; + + bdev =lookup_bdev (buf); +if (IS_ERR (bdev)) +return -ENODEV ; + + disk =get_gendisk (bdev->bd_dev, &part); + *dev =MKDEV (disk->major, disk->first_minor); +bdput (bdev); + +return 0 ; +} + +static int policy_parse_and_set (char *buf,struct io_policy_node *newpn) +{ +char *s[3 ], *p; +int ret; +int i =0 ; + +memset (s,0 ,sizeof (s)); +while ((p =strsep (&buf,":" )) !=NULL ) { +if (!*p) +continue ; + s[i++] = p; + } + + ret =devname_to_devnum (s[0 ], &newpn->dev); +if (ret) +return ret; + +strcpy (newpn->dev_name, s[0 ]); + +if (s[1 ] ==NULL ) +return -EINVAL ; + + ret =strict_strtoul (s[1 ],10 , &newpn->weight); +if (ret || newpn->weight >WEIGHT_MAX ) +return -EINVAL ; + +if (s[2 ] ==NULL ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &newpn->ioprio_class); +if (ret || newpn->ioprio_class <IOPRIO_CLASS_RT || + newpn->ioprio_class >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + +return 0 ; +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct io_policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->dev); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->dev == newpn->dev) { +if (newpn->weight) { + iog->entity.new_weight = newpn->weight; + iog->entity.new_ioprio_class = + newpn->ioprio_class; +/* +* iog weight and ioprio_class updating +* actually happens if ioprio_changed is set. +* So ensure ioprio_changed is not set until +* new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.new_weight = iocg->weight; + iog->entity.new_ioprio_class = + iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } + } + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +struct cftype bfqio_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,
@@ -1592,6 +1824,7 @@ struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_DEFAULT_GRP_WEIGHT ; iocg->ioprio_class =IO_DEFAULT_GRP_CLASS ; +INIT_LIST_HEAD (&iocg->policy_list);return &iocg->css; }
@@ -1750,6 +1983,7 @@ void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)unsigned long flags, flags1;int queue_lock_held =0 ;struct elv_fq_data *efqd; +struct io_policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained
@@ -1823,6 +2057,12 @@ locked:BUG_ON (!hlist_empty (&iocg->group_data));free_css_id (&io_subsys, &iocg->css); + +list_for_each_entry_safe (pn, pntmp, &iocg->policy_list, node) { +policy_delete_node (pn); +kfree (pn); + } +kfree (iocg); }
block/elevator-fq.h +12 -0
@@ -250,9 +250,18 @@ struct io_group {#ifdef CONFIG_DEBUG_GROUP_IOSCHED unsigned short iocg_id; +dev_t dev;#endif }; +struct io_policy_node { +struct list_head node; +char dev_name[32 ]; +dev_t dev; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup.
@@ -269,6 +278,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of io_policy_node */ +struct list_head policy_list; +spinlock_t lock;struct hlist_head group_data; };
Signature · 1 line
1.5.4.rc3
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Quoted 42 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Configure weight=500 ioprio_class=1 on
/dev/hdain this cgroup# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2Remove the policy for
/dev/hdain this cgroup# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Changelog (v1 -> v2)
- Rename some structures
- Usespin_lock_irqsave()andspin_lock_irqrestore()version to prevent from enabling the interrupts unconditionally.
- Fix policy setup bug when switching to another io scheduler.
- If a policy is available for a specific device, don't update weight and io class when writing "weight" and "iprio_class".
- Fix a bug when parsing policy string.
Quoted 2 lines
Signed-off-by Gui Jianfeng <[email protected]>---
Thanks a lot Gui. This patch seems to be working fine for me now. I will continue to do more testing and let you know if there are more issues. I will include it in next posting (V3).
Thanks
Vivek
Quoted 403 lines
block/elevator-fq.c | 258 +++++++++++++++++++++++++++++++++++++++++++++++++-- block/elevator-fq.h | 12 +++ 2 files changed, 261 insertions(+), 9 deletions(-)block/elevator-fq.c +249
-9@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -1045,12 +1048,31 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct io_policy_node *policy_search_node (const struct io_cgroup *iocg, +dev_t dev); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +dev_t dev) {struct io_entity *entity = &iog->entity; +struct io_policy_node *pn; +unsigned long flags; + +spin_lock_irqsave (&iocg->lock, flags); + pn =policy_search_node (iocg, dev); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irqrestore (&iocg->lock, flags); - entity->weight = entity->new_weight = iocg->weight; - entity->ioprio_class = entity->new_ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sched_data = &iog->sched_data; }@@ -1114,6 +1136,7 @@ static int io_cgroup_##__VAR##_write(struct cgroup *cgroup, \struct io_cgroup *iocg; \struct io_group *iog; \struct hlist_node *n; \ +struct io_policy_node *pn; \ \if (val < (__MIN) || val > (__MAX)) \return -EINVAL ; \@@ -1126,6 +1149,9 @@ static int io_cgroup_##__VAR##_write(struct cgroup *cgroup, \spin_lock_irq (&iocg->lock); \ iocg->__VAR = (unsigned long )val; \ hlist_for_each_entry(iog, n, &iocg->group_data, group_node) { \ + pn =policy_search_node (iocg, iog->dev); \ +if (pn) \ +continue ; \ iog->entity.new_##__VAR = (unsigned long)val; \smp_wmb (); \ iog->entity.ioprio_changed =1 ; \@@ -1237,7 +1263,7 @@ static u64 io_cgroup_disk_sectors_read(struct cgroup *cgroup,* to the root has already an allocated group on @bfqd. */ struct io_group *io_group_chain_alloc (struct request_queue *q,void *key, -struct cgroup *cgroup) +struct cgroup *cgroup,struct bio *bio) {struct io_cgroup *iocg;struct io_group *iog, *leaf =NULL , *prev =NULL ;@@ -1263,12 +1289,17 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,atomic_set (&iog->ref,0 ); iog->deleting =0 ; -io_group_init_entity (iocg, iog); - iog->my_entity = &iog->entity;#ifdef CONFIG_DEBUG_GROUP_IOSCHED iog->iocg_id =css_id (&iocg->css); +if (bio) { +struct gendisk *disk = bio->bi_bdev->bd_disk; + iog->dev =MKDEV (disk->major, disk->first_minor); + }#endif +io_group_init_entity (iocg, iog, iog->dev); + iog->my_entity = &iog->entity; +blk_init_request_list (&iog->rl);if (leaf ==NULL ) {@@ -1379,7 +1410,7 @@ void io_group_chain_link(struct request_queue *q, void *key,*/ struct io_group *io_find_alloc_group (struct request_queue *q,struct cgroup *cgroup,struct elv_fq_data *efqd, -int create) +int create,struct bio *bio) {struct io_cgroup *iocg =cgroup_to_io_cgroup (cgroup);struct io_group *iog =NULL ;@@ -1390,7 +1421,7 @@ struct io_group *io_find_alloc_group(struct request_queue *q,if (iog !=NULL || !create)return iog; - iog =io_group_chain_alloc (q, key, cgroup); + iog =io_group_chain_alloc (q, key, cgroup, bio);if (iog !=NULL )io_group_chain_link (q, key, cgroup, iog, efqd);@@ -1489,7 +1520,7 @@ struct io_group *io_get_io_group_bio(struct request_queue *q, struct bio *bio, goto out; } - iog =io_find_alloc_group (q, cgroup, efqd, create); + iog =io_find_alloc_group (q, cgroup, efqd, create, bio);if (!iog) {if (create) iog = efqd->root_group;@@ -1549,8 +1580,209 @@ struct io_group *io_alloc_root_group(struct request_queue *q,return iog; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct io_policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->policy_list)) + goto out; + +seq_printf (m,"dev weight class\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->policy_list, node) { +seq_printf (m,"%s %lu %lu\n" , pn->dev_name, + pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct io_policy_node *pn) +{ +list_add (&pn->node, &iocg->policy_list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct io_policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct io_policy_node *policy_search_node (const struct io_cgroup *iocg, +dev_t dev) +{ +struct io_policy_node *pn; + +if (list_empty (&iocg->policy_list)) +return NULL ; + +list_for_each_entry (pn, &iocg->policy_list, node) { +if (pn->dev == dev) +return pn; + } + +return NULL ; +} + +static int devname_to_devnum (const char *buf,dev_t *dev) +{ +struct block_device *bdev; +struct gendisk *disk; +int part; + + bdev =lookup_bdev (buf); +if (IS_ERR (bdev)) +return -ENODEV ; + + disk =get_gendisk (bdev->bd_dev, &part); + *dev =MKDEV (disk->major, disk->first_minor); +bdput (bdev); + +return 0 ; +} + +static int policy_parse_and_set (char *buf,struct io_policy_node *newpn) +{ +char *s[3 ], *p; +int ret; +int i =0 ; + +memset (s,0 ,sizeof (s)); +while ((p =strsep (&buf,":" )) !=NULL ) { +if (!*p) +continue ; + s[i++] = p; + } + + ret =devname_to_devnum (s[0 ], &newpn->dev); +if (ret) +return ret; + +strcpy (newpn->dev_name, s[0 ]); + +if (s[1 ] ==NULL ) +return -EINVAL ; + + ret =strict_strtoul (s[1 ],10 , &newpn->weight); +if (ret || newpn->weight >WEIGHT_MAX ) +return -EINVAL ; + +if (s[2 ] ==NULL ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &newpn->ioprio_class); +if (ret || newpn->ioprio_class <IOPRIO_CLASS_RT || + newpn->ioprio_class >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + +return 0 ; +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct io_policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->dev); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->dev == newpn->dev) { +if (newpn->weight) { + iog->entity.new_weight = newpn->weight; + iog->entity.new_ioprio_class = + newpn->ioprio_class; +/* +* iog weight and ioprio_class updating +* actually happens if ioprio_changed is set. +* So ensure ioprio_changed is not set until +* new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.new_weight = iocg->weight; + iog->entity.new_ioprio_class = + iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } + } + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +struct cftype bfqio_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,@@ -1592,6 +1824,7 @@ struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_DEFAULT_GRP_WEIGHT ; iocg->ioprio_class =IO_DEFAULT_GRP_CLASS ; +INIT_LIST_HEAD (&iocg->policy_list);return &iocg->css; }@@ -1750,6 +1983,7 @@ void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)unsigned long flags, flags1;int queue_lock_held =0 ;struct elv_fq_data *efqd; +struct io_policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained @@ -1823,6 +2057,12 @@ locked:BUG_ON (!hlist_empty (&iocg->group_data));free_css_id (&io_subsys, &iocg->css); + +list_for_each_entry_safe (pn, pntmp, &iocg->policy_list, node) { +policy_delete_node (pn); +kfree (pn); + } +kfree (iocg); }block/elevator-fq.h +12
-0@@ -250,9 +250,18 @@ struct io_group {#ifdef CONFIG_DEBUG_GROUP_IOSCHED unsigned short iocg_id; +dev_t dev;#endif }; +struct io_policy_node { +struct list_head node; +char dev_name[32 ]; +dev_t dev; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup. @@ -269,6 +278,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of io_policy_node */ +struct list_head policy_list; +spinlock_t lock;struct hlist_head group_data; };Signature · 1 line
1.5.4.rc3
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Quoted 42 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Configure weight=500 ioprio_class=1 on
/dev/hdain this cgroup# echo /dev/hda:500:1 > io.policy # cat io.policy dev weight class /dev/hda 500 1 /dev/hdb 300 2Remove the policy for
/dev/hdain this cgroup# echo /dev/hda:0:1 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Changelog (v1 -> v2)
- Rename some structures
- Usespin_lock_irqsave()andspin_lock_irqrestore()version to prevent from enabling the interrupts unconditionally.
- Fix policy setup bug when switching to another io scheduler.
- If a policy is available for a specific device, don't update weight and io class when writing "weight" and "iprio_class".
- Fix a bug when parsing policy string.
Quoted 2 lines
Signed-off-by Gui Jianfeng <[email protected]>---
Thanks a lot Gui. This patch seems to be working fine for me now. I will continue to do more testing and let you know if there are more issues. I will include it in next posting (V3).
Thanks
Vivek
Quoted 403 lines
block/elevator-fq.c | 258 +++++++++++++++++++++++++++++++++++++++++++++++++-- block/elevator-fq.h | 12 +++ 2 files changed, 261 insertions(+), 9 deletions(-)block/elevator-fq.c +249
-9@@ -12,6 +12,9 @@#include "elevator-fq.h" #include <linux/blktrace_api.h> #include <linux/biotrack.h> +#include <linux/seq_file.h> +#include <linux/genhd.h> +/* Values taken from cfq */ const int elv_slice_sync =HZ /10 ;@@ -1045,12 +1048,31 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); -void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct io_policy_node *policy_search_node (const struct io_cgroup *iocg, +dev_t dev); + +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog, +dev_t dev) {struct io_entity *entity = &iog->entity; +struct io_policy_node *pn; +unsigned long flags; + +spin_lock_irqsave (&iocg->lock, flags); + pn =policy_search_node (iocg, dev); +if (pn) { + entity->weight = pn->weight; + entity->new_weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + entity->new_ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->new_weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + entity->new_ioprio_class = iocg->ioprio_class; + } +spin_unlock_irqrestore (&iocg->lock, flags); - entity->weight = entity->new_weight = iocg->weight; - entity->ioprio_class = entity->new_ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sched_data = &iog->sched_data; }@@ -1114,6 +1136,7 @@ static int io_cgroup_##__VAR##_write(struct cgroup *cgroup, \struct io_cgroup *iocg; \struct io_group *iog; \struct hlist_node *n; \ +struct io_policy_node *pn; \ \if (val < (__MIN) || val > (__MAX)) \return -EINVAL ; \@@ -1126,6 +1149,9 @@ static int io_cgroup_##__VAR##_write(struct cgroup *cgroup, \spin_lock_irq (&iocg->lock); \ iocg->__VAR = (unsigned long )val; \ hlist_for_each_entry(iog, n, &iocg->group_data, group_node) { \ + pn =policy_search_node (iocg, iog->dev); \ +if (pn) \ +continue ; \ iog->entity.new_##__VAR = (unsigned long)val; \smp_wmb (); \ iog->entity.ioprio_changed =1 ; \@@ -1237,7 +1263,7 @@ static u64 io_cgroup_disk_sectors_read(struct cgroup *cgroup,* to the root has already an allocated group on @bfqd. */ struct io_group *io_group_chain_alloc (struct request_queue *q,void *key, -struct cgroup *cgroup) +struct cgroup *cgroup,struct bio *bio) {struct io_cgroup *iocg;struct io_group *iog, *leaf =NULL , *prev =NULL ;@@ -1263,12 +1289,17 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,atomic_set (&iog->ref,0 ); iog->deleting =0 ; -io_group_init_entity (iocg, iog); - iog->my_entity = &iog->entity;#ifdef CONFIG_DEBUG_GROUP_IOSCHED iog->iocg_id =css_id (&iocg->css); +if (bio) { +struct gendisk *disk = bio->bi_bdev->bd_disk; + iog->dev =MKDEV (disk->major, disk->first_minor); + }#endif +io_group_init_entity (iocg, iog, iog->dev); + iog->my_entity = &iog->entity; +blk_init_request_list (&iog->rl);if (leaf ==NULL ) {@@ -1379,7 +1410,7 @@ void io_group_chain_link(struct request_queue *q, void *key,*/ struct io_group *io_find_alloc_group (struct request_queue *q,struct cgroup *cgroup,struct elv_fq_data *efqd, -int create) +int create,struct bio *bio) {struct io_cgroup *iocg =cgroup_to_io_cgroup (cgroup);struct io_group *iog =NULL ;@@ -1390,7 +1421,7 @@ struct io_group *io_find_alloc_group(struct request_queue *q,if (iog !=NULL || !create)return iog; - iog =io_group_chain_alloc (q, key, cgroup); + iog =io_group_chain_alloc (q, key, cgroup, bio);if (iog !=NULL )io_group_chain_link (q, key, cgroup, iog, efqd);@@ -1489,7 +1520,7 @@ struct io_group *io_get_io_group_bio(struct request_queue *q, struct bio *bio, goto out; } - iog =io_find_alloc_group (q, cgroup, efqd, create); + iog =io_find_alloc_group (q, cgroup, efqd, create, bio);if (!iog) {if (create) iog = efqd->root_group;@@ -1549,8 +1580,209 @@ struct io_group *io_alloc_root_group(struct request_queue *q,return iog; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct io_policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->policy_list)) + goto out; + +seq_printf (m,"dev weight class\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->policy_list, node) { +seq_printf (m,"%s %lu %lu\n" , pn->dev_name, + pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct io_policy_node *pn) +{ +list_add (&pn->node, &iocg->policy_list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct io_policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct io_policy_node *policy_search_node (const struct io_cgroup *iocg, +dev_t dev) +{ +struct io_policy_node *pn; + +if (list_empty (&iocg->policy_list)) +return NULL ; + +list_for_each_entry (pn, &iocg->policy_list, node) { +if (pn->dev == dev) +return pn; + } + +return NULL ; +} + +static int devname_to_devnum (const char *buf,dev_t *dev) +{ +struct block_device *bdev; +struct gendisk *disk; +int part; + + bdev =lookup_bdev (buf); +if (IS_ERR (bdev)) +return -ENODEV ; + + disk =get_gendisk (bdev->bd_dev, &part); + *dev =MKDEV (disk->major, disk->first_minor); +bdput (bdev); + +return 0 ; +} + +static int policy_parse_and_set (char *buf,struct io_policy_node *newpn) +{ +char *s[3 ], *p; +int ret; +int i =0 ; + +memset (s,0 ,sizeof (s)); +while ((p =strsep (&buf,":" )) !=NULL ) { +if (!*p) +continue ; + s[i++] = p; + } + + ret =devname_to_devnum (s[0 ], &newpn->dev); +if (ret) +return ret; + +strcpy (newpn->dev_name, s[0 ]); + +if (s[1 ] ==NULL ) +return -EINVAL ; + + ret =strict_strtoul (s[1 ],10 , &newpn->weight); +if (ret || newpn->weight >WEIGHT_MAX ) +return -EINVAL ; + +if (s[2 ] ==NULL ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &newpn->ioprio_class); +if (ret || newpn->ioprio_class <IOPRIO_CLASS_RT || + newpn->ioprio_class >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + +return 0 ; +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct io_policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->dev); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->dev == newpn->dev) { +if (newpn->weight) { + iog->entity.new_weight = newpn->weight; + iog->entity.new_ioprio_class = + newpn->ioprio_class; +/* +* iog weight and ioprio_class updating +* actually happens if ioprio_changed is set. +* So ensure ioprio_changed is not set until +* new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.new_weight = iocg->weight; + iog->entity.new_ioprio_class = + iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } + } + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +struct cftype bfqio_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,@@ -1592,6 +1824,7 @@ struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_DEFAULT_GRP_WEIGHT ; iocg->ioprio_class =IO_DEFAULT_GRP_CLASS ; +INIT_LIST_HEAD (&iocg->policy_list);return &iocg->css; }@@ -1750,6 +1983,7 @@ void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)unsigned long flags, flags1;int queue_lock_held =0 ;struct elv_fq_data *efqd; +struct io_policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained @@ -1823,6 +2057,12 @@ locked:BUG_ON (!hlist_empty (&iocg->group_data));free_css_id (&io_subsys, &iocg->css); + +list_for_each_entry_safe (pn, pntmp, &iocg->policy_list, node) { +policy_delete_node (pn); +kfree (pn); + } +kfree (iocg); }block/elevator-fq.h +12
-0@@ -250,9 +250,18 @@ struct io_group {#ifdef CONFIG_DEBUG_GROUP_IOSCHED unsigned short iocg_id; +dev_t dev;#endif }; +struct io_policy_node { +struct list_head node; +char dev_name[32 ]; +dev_t dev; +unsigned long weight; +unsigned long ioprio_class; +}; +/** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup. @@ -269,6 +278,9 @@ struct io_cgroup {unsigned long weight, ioprio_class; +/* list of io_policy_node */ +struct list_head policy_list; +spinlock_t lock;struct hlist_head group_data; };Signature · 1 line
1.5.4.rc3
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to a message not in the archiveHi Gui,
Quoted 13 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Quoted 6 lines
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2
Users can specify a device file of a partition for io.policy. In this case, io_policy_node::dev_name is set as a name of the partition device like /dev/sda2.
ex)
# cd /mnt/cgroup
# cat /dev/sda2:500:2 > io.policy
# echo io.policy
dev weight class
/dev/sda2 500 2I believe io_policy_node::dev_name should be set a generic device name like /dev/sda.
What do you think about it?
--- block/elevator-fq.c | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-)
block/elevator-fq.c +6 -1
@@ -1631,11 +1631,12 @@ static struct io_policy_node *policy_search_node(const struct io_cgroup *iocg,return NULL ; } -static int devname_to_devnum (const char *buf,dev_t *dev) +static int devname_to_devnum (char *buf,dev_t *dev) {struct block_device *bdev;struct gendisk *disk;int part; +char *c; bdev =lookup_bdev (buf);if (IS_ERR (bdev))
@@ -1645,6 +1646,10 @@ static int devname_to_devnum(const char *buf, dev_t *dev) *dev =MKDEV (disk->major, disk->first_minor);bdput (bdev); + c =strrchr (buf,'/' ); +if (c) +strcpy (c+1 , disk->disk_name); +return 0 ; }
Signature · 7 lines
1.5.4.3
--
IKEDA, Munehiro
NEC Corporation of America
m-ikeda-MDRzhb/z0dd8UrSeD/[email protected]Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
In reply to Gui JianfengHi Gui,
Quoted 11 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling.
A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you
don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Quoted 6 lines
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2
Users can specify a device file of a partition for io.policy. In this case, io_policy_node::dev_name is set as a name of the partition device like /dev/sda2.
ex)
# cd /mnt/cgroup # cat /dev/sda2:500:2 > io.policy # echo io.policy dev weight class /dev/sda2 500 2
I believe io_policy_node::dev_name should be set a generic device name like /dev/sda.
What do you think about it?
---block/elevator-fq.c | 7 ++++++-
1 files changed, 6 insertions(+), 1 deletions(-)
block/elevator-fq.c +0 -0
@@ -1631,11 +1631,12 @@ static struct io_policy_node *policy_search_node(const struct io_cgroup *iocg,
return NULL; }
-static int devname_to_devnum(const char *buf, dev_t *dev) +static int devname_to_devnum(char *buf, dev_t *dev)
{
struct block_device *bdev;
struct gendisk *disk;
int part;+ char *c;
bdev = lookup_bdev(buf); if (IS_ERR(bdev)) @@ -1645,6 +1646,10 @@ static int devname_to_devnum(const char *buf, dev_t *dev) *dev = MKDEV(disk->major, disk->first_minor); bdput(bdev);
+ c = strrchr(buf, '/');
+ if (c)
+ strcpy(c+1, disk->disk_name);
+
return 0; }
Signature · 7 lines
1.5.4.3 -- IKEDA, Munehiro NEC Corporation of America [email protected]
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Quoted 37 lines · Munehiro
Hi Gui,
Quoted 16 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling.
A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup.
The original "weight" and "ioprio_class" files are still available. If you
don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Quoted 6 lines
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Users can specify a device file of a partition for io.policy. In this case,
io_policy_node::dev_nameis set as a name of the partition device like/dev/sda2.ex)
# cd /mnt/cgroup # cat /dev/sda2:500:2 > io.policy # echo io.policy dev weight class /dev/sda2 500 2
Quoted 3 lines
I believe
io_policy_node::dev_nameshould be set a generic device name like/dev/sda.
What do you think about it?
Hi Ikeda-san,
Sorry for the late reply. Thanks for pointing this out. yes, it does the right thing but shows a wrong name. IMHO, Inputing a sigle partition should not be allowed since the policy is disk basis. So how about the following patch?
---
block/elevator-fq.c +3 -0
@@ -1650,6 +1650,9 @@ static int devname_to_devnum(const char *buf, dev_t *dev)return -ENODEV ; disk =get_gendisk (bdev->bd_dev, &part); +if (part) +return -EINVAL ; + *dev =MKDEV (disk->major, disk->first_minor);bdput (bdev);
Quoted 37 lines
Signed-off-by Munehiro "Muuhh" Ikeda <[email protected]>---
block/elevator-fq.c| 7 ++++++-1 files changed, 6 insertions(+), 1 deletions(-)block/elevator-fq.c +2
-1@@ -1631,11 +1631,12 @@ static struct io_policy_node *policy_search_node(const struct io_cgroup *iocg, return NULL; } -static int devname_to_devnum(const char *buf, dev_t *dev) +static int devname_to_devnum(char *buf, dev_t *dev) { struct block_device *bdev; struct gendisk *disk; int part; + char *c; bdev = lookup_bdev(buf);if (IS_ERR(bdev))block/elevator-fq.c +4
-0@@ -1645,6 +1646,10 @@ static int devname_to_devnum(const char *buf, dev_t *dev) *dev =MKDEV (disk->major, disk->first_minor);bdput (bdev); + c =strrchr (buf,'/' ); +if (c) +strcpy (c+1 , disk->disk_name); +return 0 ; }
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] IO Controller: Add per-device weight and ioprio_class handling
Quoted 55 lines · Gui Jianfeng
Quoted 36 lines · Munehiro
Hi Gui,
Quoted 16 lines · Gui Jianfeng
Hi Vivek,
This patch enables per-cgroup per-device weight and
ioprio_classhandling.
A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup.
The original "weight" and "ioprio_class" files are still available. If you
don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.You can use the following format to play with the new interface. #echo DEV:weight:ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for DEV.
Quoted 6 lines
Examples:
Configure weight=300 ioprio_class=2 on/dev/hdbin this cgroup# echo /dev/hdb:300:2 > io.policy # cat io.policy dev weight class /dev/hdb 300 2Users can specify a device file of a partition for io.policy. In this case,
io_policy_node::dev_nameis set as a name of the partition device like/dev/sda2.ex)
# cd /mnt/cgroup # cat /dev/sda2:500:2 > io.policy # echo io.policy dev weight class /dev/sda2 500 2Quoted 3 lines
I believe
io_policy_node::dev_nameshould be set a generic device name like/dev/sda.
What do you think about it?Hi Ikeda-san,
Sorry for the late reply. Thanks for pointing this out. yes, it does the right thing but shows a wrong name. IMHO, Inputing a sigle partition should not be allowed since the policy is disk basis. So how about the following patch?
Signed-off-by Gui Jianfeng <[email protected]>---
block/elevator-fq.c +0
-0
block/elevator-fq.c static int devname_to_devnum(const char *buf, dev_t *dev) · line 1658
Quoted 9 lines
block/elevator-fq.c +3
-0@@ -1650,6 +1650,9 @@ static int devname_to_devnum(const char *buf, dev_t *dev)return -ENODEV ; disk =get_gendisk (bdev->bd_dev, &part); +if (part) +return -EINVAL ; + *dev =MKDEV (disk->major, disk->first_minor);bdput (bdev);
It looks nicer and reasonable for me.
Thanks!
Signature · 3 lines
IKEDA, Munehiro NEC Corporation of America m-ikeda-MDRzhb/z0dd8UrSeD/[email protected]
A message that is not in the archive (sent off-list or not yet received): 4A11E244.2000305-MDRzhb/z0dd8UrSeD/[email protected]