2.6.39-rc5-git2 boot crashs

Quoted 2 lines · Linus Torvalds

Now, Werner is actually _running_ on an AMD Opteron (or whatever family 15 is), I think.

http://marc.info/?l=linux-kernel&m=130438580705332 says it's a AMD Athlon(tm) 64 X2 Dual Core Processor 4800+, i.e. a good old K8 desktop. The X86_ELAN Kconfig option, however, says:

config X86_ELAN
bool "AMD Elan"
depends on X86_32
depends on X86_EXTENDED_PLATFORM
---help---
Select this for an AMD Elan processor.

Do not use this option for K6/Athlon/Opteron processors!

If unsure, choose "PC-compatible" instead.

so if the sentence before last used to mean anything, this could be a problem. Quick search about it gives http://lkml.org/lkml/2004/1/12/239
which introduces that different compiler arch for ELAN: -march=i486, which could conflict with the generic selection?

Hmm.

Signature · 2 lines

Regards/Gruss,
    Boris.

Ingo Molnar

2011-05-04 07:39 UTC #11 lore raw

Quoted 15 lines · Borislav Petkov

Quoted 2 lines · Linus Torvalds
Now, Werner is actually _running_ on an AMD Opteron (or whatever family 15 is), I think.
http://marc.info/?l=linux-kernel&m=130438580705332 says it's a AMD Athlon(tm) 64 X2 Dual Core Processor 4800+, i.e. a good old K8 desktop. The X86_ELAN Kconfig option, however, says:
config X86_ELAN
bool "AMD Elan"
depends on X86_32
depends on X86_EXTENDED_PLATFORM
---help---
Select this for an AMD Elan processor.

Quoted 8 lines

Do not use this option for K6/Athlon/Opteron processors!
If unsure, choose "PC-compatible" instead.
so if the sentence before last used to mean anything, this could be a problem. Quick search about it gives http://lkml.org/lkml/2004/1/12/239
which introduces that different compiler arch for ELAN: -march=i486, which could conflict with the generic selection?

Well, but CONFIG_X86_ELAN=y always worked/booted fine on generic hardware - including later AMD CPUs. I have booted it on an AMD Athlon64 CPU today:

config-Wed_May__4_09_26_50_CEST_2011.good:CONFIG_X86_32_SMP=y config-Wed_May__4_09_26_50_CEST_2011.good:CONFIG_MELAN=y config-Wed_May__4_09_26_50_CEST_2011.good:CONFIG_SMP=y

When it comes to regressions it's past behavior that counts, not the Kconfig help text.

Thanks,

Ingo

Borislav Petkov

maintainer

2011-05-04 07:55 UTC #12 lore raw

Quoted 30 lines · Ingo Molnar

Quoted 15 lines · Borislav Petkov
Quoted 2 lines · Linus Torvalds
Now, Werner is actually _running_ on an AMD Opteron (or whatever family 15 is), I think.
http://marc.info/?l=linux-kernel&m=130438580705332 says it's a AMD Athlon(tm) 64 X2 Dual Core Processor 4800+, i.e. a good old K8 desktop. The X86_ELAN Kconfig option, however, says:
config X86_ELAN
bool "AMD Elan"
depends on X86_32
depends on X86_EXTENDED_PLATFORM
---help---
Select this for an AMD Elan processor.
Quoted 8 lines
Do not use this option for K6/Athlon/Opteron processors!
If unsure, choose "PC-compatible" instead.
so if the sentence before last used to mean anything, this could be a problem. Quick search about it gives http://lkml.org/lkml/2004/1/12/239
which introduces that different compiler arch for ELAN: -march=i486, which could conflict with the generic selection?
Well, but CONFIG_X86_ELAN=y always worked/booted fine on generic hardware - including later AMD CPUs. I have booted it on an AMD Athlon64 CPU today:

Quoted 6 lines

config-Wed_May__4_09_26_50_CEST_2011.good:CONFIG_X86_32_SMP=y config-Wed_May__4_09_26_50_CEST_2011.good:CONFIG_MELAN=y config-Wed_May__4_09_26_50_CEST_2011.good:CONFIG_SMP=y
When it comes to regressions it's past behavior that counts, not the Kconfig help text.

Right, I was referring to the odd problem Linus was talking about:

"He had CONFIG_X86_EXTENDED_PLATTFORM and CONFIG_X86_ELAN on, and that has apparently worked for him (and a lot of other people - he does a distribution) up until 2.6.38. But as of 2.6.39-rc1 it causes some really odd problems under IO (his test-case is "unzip", but that's probably fairly random). The problem seems to show up as a bogus IO list for SATA, causing a big WARN_ON() or oops and then a dead machine due to IO problems."

So booting might've not triggered it. But reportedly .38 was fine so yeah, the Kconfig help text might not even mean anything anymore.

Maybe I should run the same .config and test case on a K8 box here to see what happens.

@Werner: can you send me the exact .config and the testcase that triggers the issue?

Thanks.

Signature · 2 lines

Regards/Gruss,
    Boris.

Ingo Molnar

2011-05-04 08:37 UTC #13 lore raw

Quoted 7 lines · Linus Torvalds

Quoted 4 lines · Ingo Molnar
I have no smart ideas straight away - just an observation: i keep testing CONFIG_X86_ELAN=y on real hardware, and it's enabled in about 4% of my configs:
So how often do you do more than just boot?

Not very often - but 'to boot' means a certain amount of filesystem work - and so does the 'prepare to boot the next kernel' step.

So i took Werner's .config.zipproblem and modified it to make it bootable:
removed CONFIG_ROOT_NFS=y and disabled CONFIG_IDE - both of which keep my box from booting. I've attached an updated .config.zipproblem2 file: Werner, can you confirm that this still fails for you?

So i booted v2.6.39-rc5-254-g5933f2a on an AMD box (which is SMP in fact, so should trigger races even faster):

Kernel 2.6.39-rc5-i486-1sys+ on an i686

and started a couple of such IO-intense loops:

  FILE=bigfile.$RANDOM;
  while sync; do rm -f $FILE; dd if=/dev/urandom of=$FILE bs=1000 count=10000; done &

this creates patterns of high IO combined with periods waiting for IO to flush:

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------

 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 8  0      0 621140   8968 346500    0    0    29   874 1044  135  0 27 71  2  0
 8  0      0 614692   8984 352904    0    0     0     0 2011  658  0 100  0  0  0
 8  0      0 608492   8992 359104    0    0     0     0 2096  709  1 99  0  0  0
 8  0      0 602016   8992 365332    0    0     0     0 2006  651  0 100  0  0  0
 8  0      0 595692   8992 371760    0    0     0     0 2005  647  0 100  0  0  0
 4  5      0 588128   9000 378080    0    0     0 31508 2031  610  0 100  0  0  0
 6  2      0 583168   9068 383664    0    0     4  7532 2256  668  0 91  0  9  0
 8  0      0 586888   9116 380212    0    0     0   456 2073  804  1 100  0  0  0
 5  3      0 620484   9408 346904    0    0     0 14932 2768 1901  0 94  1  6  0
 8  0      0 633656   9576 333392    0    0     0  6180 2369 1232  0 98  0  2  0
 8  0      0 627208   9576 339720    0    0     0     0 2004  650  0 100  0  0  0
 8  0      0 621008   9580 346032    0    0     0     0 2048  680  1 100  0  0  0
 8  0      0 614436   9588 352296    0    0     0     0 2004  657  0 100  0  0  0
 8  0      0 618652   9708 348296    0    0     0 25984 2283  816  0 92  0  7  0
 8  0      0 612424   9708 354576    0    0     0     0 2005  651  0 100  0  0  0
 8  0      0 605976   9716 360892    0    0     0     0 2004  652  0 100  0  0  0
 8  0      0 599652   9716 367208    0    0     0     0 2006  654  0 100  0  0  0
 8  0      0 593204   9720 373528    0    0     0     0 2004  652  0 100  0  0  0
 7  2      0 585764   9728 379852    0    0     0 31612 2038  649  0 100  0  0  0
 8  0      0 590104   9832 376180    0    0     4  7400 2274  920  0 98  0  2  0
 2  6      0 597288  10008 368892    0    0     0  8636 2432 1373  0 95  0  5  0
 8  0      0 627180  10208 339096    0    0     0  4324 2426 1429  0 97  0  2  0
 8  0      0 630404  10284 335480    0    0     0  6408 2202  970  1 97  0  2  0

And indeed, after a couple of minutes testing i triggered this beauty:

BUG: unable to handle kernel NULL pointer dereference at 00000008
IP: [<c14e85c0>] generic_make_request+0x86/0x3f4
*pde = 00000000 
Oops: 0000 [#1] SMP 
last sysfs file: /sys/devices/pci0000:00/0000:00:0a.0/net/eth0/address
Modules linked in:

Pid: 2969, comm: flush-8:0 Not tainted 2.6.39-rc5-i486-1sys+ #122580 System manufacturer System Product Name/A8N-E
EIP: 0060:[<c14e85c0>] EFLAGS: 00010202 CPU: 1
EIP is at generic_make_request+0x86/0x3f4
EAX: 00000000 EBX: f569e280 ECX: f6a00000 EDX: f5528000
ESI: 00000008 EDI: 00000001 EBP: f5fd7c8c ESP: f5fd7c14
DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
Process flush-8:0 (pid: 2969, ti=f5fd6000 task=f64d0ab0 task.ti=f5fd6000)

Stack:
 f569e280 c10988af f5fd7c84 c1d74030 f6402400 f569e300 00081ddb f569e280
 c10988af f5fd7c84 f64d0ab0 0000007b 0000007b 000000d8 00000008 ffffffc1
 00000000 00000000 00000246 f6a06c80 055c08fe 00000000 c10988af 00000080
Call Trace:
 [<c10988af>] ? mempool_alloc_slab+0x13/0x15
 [<c1d74030>] ? common_interrupt+0x30/0x40
 [<c10988af>] ? mempool_alloc_slab+0x13/0x15
 [<c10988af>] ? mempool_alloc_slab+0x13/0x15
 [<c14e89f9>] submit_bio+0xcb/0xe4
 [<c10a8048>] ? inc_zone_page_state+0xe/0x88
 [<c10ed8b5>] ? bio_init+0x9/0x2e
 [<c10ee05b>] ? bio_alloc_bioset+0x3c/0x9c
 [<c10ea27a>] submit_bh+0xc6/0xe0
 [<c10eb8f3>] __block_write_full_page+0x20a/0x2df
 [<c10ed9d9>] ? bio_put+0x8/0x2c
 [<c10ea9e5>] ? end_buffer_async_read+0xd5/0xd5
 [<c10eba88>] block_write_full_page_endio+0xc0/0xc8
 [<c10ea9e5>] ? end_buffer_async_read+0xd5/0xd5
 [<c10ebaa7>] block_write_full_page+0x17/0x19
 [<c10ea9e5>] ? end_buffer_async_read+0xd5/0xd5
 [<c1156fd0>] ext3_ordered_writepage+0xc8/0x19c
 [<c11566ee>] ? bput_one+0x10/0x10
 [<c109c9ce>] __writepage+0x10/0x28
 [<c109cd96>] write_cache_pages+0x1c9/0x283
 [<c109c9be>] ? bdi_set_max_ratio+0x52/0x52
 [<c109ce86>] generic_writepages+0x36/0x49
 [<c109d994>] do_writepages+0x28/0x2b
 [<c10e5642>] writeback_single_inode+0xa6/0x18d
 [<c10e58ef>] writeback_sb_inodes+0xa6/0x10b
 [<c10e6267>] writeback_inodes_wb+0xd9/0xee
 [<c10e642b>] wb_writeback+0x1af/0x26d
 [<c1045c12>] ? try_to_del_timer_sync+0x81/0x89
 [<c103faca>] ? local_bh_disable+0x8/0x18
 [<c10e655a>] wb_do_writeback+0x71/0x181
 [<c1045dc0>] ? add_timer_on+0x95/0x95
 [<c1045cb1>] ? del_timer+0xc/0x86
 [<c10e66d8>] bdi_writeback_thread+0x6e/0x186
 [<c10e666a>] ? wb_do_writeback+0x181/0x181
 [<c1051f73>] kthread+0x67/0x6c
 [<c1051f0c>] ? kthread_worker_fn+0x114/0x114
 [<c1d74046>] kernel_thread_helper+0x6/0x10
Code: 00 c7 45 c8 00 00 00 00 8d 55 c8 89 90 f4 02 00 00 89 45 b0 8b 53 20 c1 ea 09 89 55 c0 e8 e3 8c 88 00 83 7d c0 00 74 69 8b 43 0c <8b> 40 08 8b 90 84 00 00 00 f6 c2 01 74 04 f3 90 eb f1 8b 70 7c 
EIP: [<c14e85c0>] generic_make_request+0x86/0x3f4 SS:ESP 0068:f5fd7c14
CR2: 0000000000000008
---[ end trace c45e837de578cd2f ]---
------------[ cut here ]------------

( the full crashlog is attached as well - hardware details can be found there
although i doubt it matters. )

Seems to be a generic IO/BDI badness at first sight. It gives me the appearance of a race or boundary condition bug, not that of memory corruption.

Here's the crashing generic_make_request() function:

c14e853a <generic_make_request>:

c14e853a:	55                   	push   %ebp
c14e853b:	89 e5                	mov    %esp,%ebp
c14e853d:	57                   	push   %edi
c14e853e:	56                   	push   %esi
c14e853f:	53                   	push   %ebx
c14e8540:	83 ec 6c             	sub    $0x6c,%esp
c14e8543:	e8 08 bb 88 00       	call   c1d74050 <mcount>
c14e8548:	89 c3                	mov    %eax,%ebx
c14e854a:	65 a1 14 00 00 00    	mov    %gs:0x14,%eax
c14e8550:	89 45 f0             	mov    %eax,-0x10(%ebp)
c14e8553:	31 c0                	xor    %eax,%eax
c14e8555:	64 a1 c4 c4 38 c2    	mov    %fs:0xc238c4c4,%eax
c14e855b:	83 b8 f4 02 00 00 00 	cmpl   $0x0,0x2f4(%eax)
c14e8562:	74 23                	je     c14e8587 <generic_make_request+0x4d>
c14e8564:	8b 80 f4 02 00 00    	mov    0x2f4(%eax),%eax
c14e856a:	c7 43 08 00 00 00 00 	movl   $0x0,0x8(%ebx)
c14e8571:	8b 50 04             	mov    0x4(%eax),%edx
c14e8574:	85 d2                	test   %edx,%edx
c14e8576:	74 05                	je     c14e857d <generic_make_request+0x43>
c14e8578:	89 5a 08             	mov    %ebx,0x8(%edx)
c14e857b:	eb 02                	jmp    c14e857f <generic_make_request+0x45>
c14e857d:	89 18                	mov    %ebx,(%eax)
c14e857f:	89 58 04             	mov    %ebx,0x4(%eax)
c14e8582:	e9 8e 03 00 00       	jmp    c14e8915 <generic_make_request+0x3db>
c14e8587:	83 7b 08 00          	cmpl   $0x0,0x8(%ebx)
c14e858b:	74 02                	je     c14e858f <generic_make_request+0x55>
c14e858d:	0f 0b                	ud2    
c14e858f:	c7 45 cc 00 00 00 00 	movl   $0x0,-0x34(%ebp)
c14e8596:	c7 45 c8 00 00 00 00 	movl   $0x0,-0x38(%ebp)
c14e859d:	8d 55 c8             	lea    -0x38(%ebp),%edx
c14e85a0:	89 90 f4 02 00 00    	mov    %edx,0x2f4(%eax)
c14e85a6:	89 45 b0             	mov    %eax,-0x50(%ebp)
c14e85a9:	8b 53 20             	mov    0x20(%ebx),%edx
c14e85ac:	c1 ea 09             	shr    $0x9,%edx
c14e85af:	89 55 c0             	mov    %edx,-0x40(%ebp)
c14e85b2:	e8 e3 8c 88 00       	call   c1d7129a <_cond_resched>
c14e85b7:	83 7d c0 00          	cmpl   $0x0,-0x40(%ebp)
c14e85bb:	74 69                	je     c14e8626 <generic_make_request+0xec>
c14e85bd:	8b 43 0c             	mov    0xc(%ebx),%eax
c14e85c0:	8b 40 08             	mov    0x8(%eax),%eax
c14e85c3:	8b 90 84 00 00 00    	mov    0x84(%eax),%edx
c14e85c9:	f6 c2 01             	test   $0x1,%dl
c14e85cc:	74 04                	je     c14e85d2 <generic_make_request+0x98>
c14e85ce:	f3 90                	pause  
c14e85d0:	eb f1                	jmp    c14e85c3 <generic_make_request+0x89>
c14e85d2:	8b 70 7c             	mov    0x7c(%eax),%esi
c14e85d5:	8b b8 80 00 00 00    	mov    0x80(%eax),%edi
c14e85db:	39 90 84 00 00 00    	cmp    %edx,0x84(%eax)
c14e85e1:	75 e0                	jne    c14e85c3 <generic_make_request+0x89>
c14e85e3:	89 f0                	mov    %esi,%eax
c14e85e5:	89 fa                	mov    %edi,%edx
c14e85e7:	0f ac d0 09          	shrd   $0x9,%edx,%eax
c14e85eb:	c1 fa 09             	sar    $0x9,%edx
c14e85ee:	89 d1                	mov    %edx,%ecx
c14e85f0:	09 c1                	or     %eax,%ecx
c14e85f2:	74 32                	je     c14e8626 <generic_make_request+0xec>
c14e85f4:	8b 0b                	mov    (%ebx),%ecx
c14e85f6:	89 4d c4             	mov    %ecx,-0x3c(%ebp)
c14e85f9:	8b 4b 04             	mov    0x4(%ebx),%ecx
c14e85fc:	8b 75 c0             	mov    -0x40(%ebp),%esi
c14e85ff:	31 ff                	xor    %edi,%edi
c14e8601:	83 fa 00             	cmp    $0x0,%edx
c14e8604:	77 09                	ja     c14e860f <generic_make_request+0xd5>
c14e8606:	3b 45 c0             	cmp    -0x40(%ebp),%eax
c14e8609:	0f 82 21 02 00 00    	jb     c14e8830 <generic_make_request+0x2f6>
c14e860f:	29 f0                	sub    %esi,%eax
c14e8611:	19 fa                	sbb    %edi,%edx
c14e8613:	39 ca                	cmp    %ecx,%edx
c14e8615:	77 0f                	ja     c14e8626 <generic_make_request+0xec>
c14e8617:	0f 82 13 02 00 00    	jb     c14e8830 <generic_make_request+0x2f6>
c14e861d:	3b 45 c4             	cmp    -0x3c(%ebp),%eax
c14e8620:	0f 82 0a 02 00 00    	jb     c14e8830 <generic_make_request+0x2f6>
c14e8626:	c7 45 b4 00 00 00 00 	movl   $0x0,-0x4c(%ebp)
c14e862d:	c7 45 b8 ff ff ff ff 	movl   $0xffffffff,-0x48(%ebp)
c14e8634:	c7 45 bc ff ff ff ff 	movl   $0xffffffff,-0x44(%ebp)
c14e863b:	8b 45 c0             	mov    -0x40(%ebp),%eax
c14e863e:	89 45 9c             	mov    %eax,-0x64(%ebp)
c14e8641:	c7 45 a0 00 00 00 00 	movl   $0x0,-0x60(%ebp)
c14e8648:	8b 43 0c             	mov    0xc(%ebx),%eax
c14e864b:	89 45 98             	mov    %eax,-0x68(%ebp)
c14e864e:	89 c2                	mov    %eax,%edx
c14e8650:	8b 40 58             	mov    0x58(%eax),%eax
c14e8653:	8b 80 c8 01 00 00    	mov    0x1c8(%eax),%eax
c14e8659:	89 45 c4             	mov    %eax,-0x3c(%ebp)
c14e865c:	85 c0                	test   %eax,%eax
c14e865e:	75 33                	jne    c14e8693 <generic_make_request+0x159>
c14e8660:	89 d1                	mov    %edx,%ecx
c14e8662:	8b 33                	mov    (%ebx),%esi
c14e8664:	8b 7b 04             	mov    0x4(%ebx),%edi
c14e8667:	8d 55 d0             	lea    -0x30(%ebp),%edx
c14e866a:	89 c8                	mov    %ecx,%eax
c14e866c:	e8 27 8c c2 ff       	call   c1111298 <bdevname>
c14e8671:	89 74 24 08          	mov    %esi,0x8(%esp)
c14e8675:	89 7c 24 0c          	mov    %edi,0xc(%esp)
c14e8679:	89 44 24 04          	mov    %eax,0x4(%esp)
c14e867d:	c7 04 24 52 b5 01 c2 	movl   $0xc201b552,(%esp)
c14e8684:	e8 79 17 87 00       	call   c1d59e02 <printk>
c14e8689:	ba fb ff ff ff       	mov    $0xfffffffb,%edx
c14e868e:	e9 43 02 00 00       	jmp    c14e88d6 <generic_make_request+0x39c>
c14e8693:	f6 43 14 40          	testb  $0x40,0x14(%ebx)
c14e8697:	75 36                	jne    c14e86cf <generic_make_request+0x195>
c14e8699:	8b 4d c4             	mov    -0x3c(%ebp),%ecx
c14e869c:	8b b9 3c 02 00 00    	mov    0x23c(%ecx),%edi
c14e86a2:	39 7d c0             	cmp    %edi,-0x40(%ebp)
c14e86a5:	76 28                	jbe    c14e86cf <generic_make_request+0x195>
c14e86a7:	8b 4d 98             	mov    -0x68(%ebp),%ecx
c14e86aa:	8b 73 20             	mov    0x20(%ebx),%esi
c14e86ad:	c1 ee 09             	shr    $0x9,%esi
c14e86b0:	8d 55 d0             	lea    -0x30(%ebp),%edx
c14e86b3:	89 c8                	mov    %ecx,%eax
c14e86b5:	e8 de 8b c2 ff       	call   c1111298 <bdevname>
c14e86ba:	89 7c 24 0c          	mov    %edi,0xc(%esp)
c14e86be:	89 74 24 08          	mov    %esi,0x8(%esp)
c14e86c2:	89 44 24 04          	mov    %eax,0x4(%esp)
c14e86c6:	c7 04 24 9f b5 01 c2 	movl   $0xc201b59f,(%esp)
c14e86cd:	eb b5                	jmp    c14e8684 <generic_make_request+0x14a>
c14e86cf:	8b 55 c4             	mov    -0x3c(%ebp),%edx
c14e86d2:	8b 82 a4 01 00 00    	mov    0x1a4(%edx),%eax
c14e86d8:	a8 20                	test   $0x20,%al
c14e86da:	75 ad                	jne    c14e8689 <generic_make_request+0x14f>
c14e86dc:	8b 43 20             	mov    0x20(%ebx),%eax
c14e86df:	c1 e8 09             	shr    $0x9,%eax
c14e86e2:	0f 84 80 00 00 00    	je     c14e8768 <generic_make_request+0x22e>
c14e86e8:	8b 45 98             	mov    -0x68(%ebp),%eax
c14e86eb:	3b 40 44             	cmp    0x44(%eax),%eax
c14e86ee:	74 78                	je     c14e8768 <generic_make_request+0x22e>
c14e86f0:	8b 50 4c             	mov    0x4c(%eax),%edx
c14e86f3:	89 55 a8             	mov    %edx,-0x58(%ebp)
c14e86f6:	8b 32                	mov    (%edx),%esi
c14e86f8:	8b 7a 04             	mov    0x4(%edx),%edi
c14e86fb:	03 33                	add    (%ebx),%esi
c14e86fd:	13 7b 04             	adc    0x4(%ebx),%edi
c14e8700:	89 33                	mov    %esi,(%ebx)
c14e8702:	89 7b 04             	mov    %edi,0x4(%ebx)
c14e8705:	8b 40 44             	mov    0x44(%eax),%eax
c14e8708:	89 45 a4             	mov    %eax,-0x5c(%ebp)
c14e870b:	89 43 0c             	mov    %eax,0xc(%ebx)
c14e870e:	89 f2                	mov    %esi,%edx
c14e8710:	89 f9                	mov    %edi,%ecx
c14e8712:	8b 45 a8             	mov    -0x58(%ebp),%eax
c14e8715:	2b 10                	sub    (%eax),%edx
c14e8717:	1b 48 04             	sbb    0x4(%eax),%ecx
c14e871a:	89 55 a8             	mov    %edx,-0x58(%ebp)
c14e871d:	89 4d ac             	mov    %ecx,-0x54(%ebp)
c14e8720:	8b 45 98             	mov    -0x68(%ebp),%eax
c14e8723:	8b 38                	mov    (%eax),%edi
c14e8725:	8b 55 a4             	mov    -0x5c(%ebp),%edx
c14e8728:	8b 42 58             	mov    0x58(%edx),%eax
c14e872b:	8b 80 c8 01 00 00    	mov    0x1c8(%eax),%eax
c14e8731:	89 45 a4             	mov    %eax,-0x5c(%ebp)
c14e8734:	e9 00 00 00 00       	jmp    c14e8739 <generic_make_request+0x1ff>
c14e8739:	eb 2d                	jmp    c14e8768 <generic_make_request+0x22e>
c14e873b:	8b 35 84 4e 2a c2    	mov    0xc22a4e84,%esi
c14e8741:	85 f6                	test   %esi,%esi
c14e8743:	74 23                	je     c14e8768 <generic_make_request+0x22e>
c14e8745:	8b 46 04             	mov    0x4(%esi),%eax
c14e8748:	8b 55 a8             	mov    -0x58(%ebp),%edx
c14e874b:	8b 4d ac             	mov    -0x54(%ebp),%ecx
c14e874e:	89 54 24 04          	mov    %edx,0x4(%esp)
c14e8752:	89 4c 24 08          	mov    %ecx,0x8(%esp)
c14e8756:	89 3c 24             	mov    %edi,(%esp)
c14e8759:	89 d9                	mov    %ebx,%ecx
c14e875b:	8b 55 a4             	mov    -0x5c(%ebp),%edx
c14e875e:	ff 16                	call   *(%esi)
c14e8760:	83 c6 08             	add    $0x8,%esi
c14e8763:	83 3e 00             	cmpl   $0x0,(%esi)
c14e8766:	eb db                	jmp    c14e8743 <generic_make_request+0x209>
c14e8768:	89 d8                	mov    %ebx,%eax
c14e876a:	e8 a1 a2 c0 ff       	call   c10f2a10 <bio_integrity_enabled>
c14e876f:	85 c0                	test   %eax,%eax
c14e8771:	74 0f                	je     c14e8782 <generic_make_request+0x248>
c14e8773:	89 d8                	mov    %ebx,%eax
c14e8775:	e8 06 a4 c0 ff       	call   c10f2b80 <bio_integrity_prep>
c14e877a:	85 c0                	test   %eax,%eax
c14e877c:	0f 85 07 ff ff ff    	jne    c14e8689 <generic_make_request+0x14f>
c14e8782:	83 7d bc ff          	cmpl   $0xffffffff,-0x44(%ebp)
c14e8786:	75 06                	jne    c14e878e <generic_make_request+0x254>
c14e8788:	83 7d b8 ff          	cmpl   $0xffffffff,-0x48(%ebp)
c14e878c:	74 37                	je     c14e87c5 <generic_make_request+0x28b>
c14e878e:	e9 00 00 00 00       	jmp    c14e8793 <generic_make_request+0x259>
c14e8793:	eb 30                	jmp    c14e87c5 <generic_make_request+0x28b>
c14e8795:	8b 35 84 4e 2a c2    	mov    0xc22a4e84,%esi
c14e879b:	85 f6                	test   %esi,%esi
c14e879d:	74 26                	je     c14e87c5 <generic_make_request+0x28b>
c14e879f:	8b 46 04             	mov    0x4(%esi),%eax
c14e87a2:	8b 55 b8             	mov    -0x48(%ebp),%edx
c14e87a5:	8b 4d bc             	mov    -0x44(%ebp),%ecx
c14e87a8:	89 54 24 04          	mov    %edx,0x4(%esp)
c14e87ac:	89 4c 24 08          	mov    %ecx,0x8(%esp)
c14e87b0:	8b 4d b4             	mov    -0x4c(%ebp),%ecx
c14e87b3:	89 0c 24             	mov    %ecx,(%esp)
c14e87b6:	89 d9                	mov    %ebx,%ecx
c14e87b8:	8b 55 c4             	mov    -0x3c(%ebp),%edx
c14e87bb:	ff 16                	call   *(%esi)
c14e87bd:	83 c6 08             	add    $0x8,%esi
c14e87c0:	83 3e 00             	cmpl   $0x0,(%esi)
c14e87c3:	eb d8                	jmp    c14e879d <generic_make_request+0x263>
c14e87c5:	8b 03                	mov    (%ebx),%eax
c14e87c7:	8b 53 04             	mov    0x4(%ebx),%edx
c14e87ca:	89 45 b8             	mov    %eax,-0x48(%ebp)
c14e87cd:	89 55 bc             	mov    %edx,-0x44(%ebp)
c14e87d0:	8b 43 0c             	mov    0xc(%ebx),%eax
c14e87d3:	8b 10                	mov    (%eax),%edx
c14e87d5:	89 55 b4             	mov    %edx,-0x4c(%ebp)
c14e87d8:	83 7d c0 00          	cmpl   $0x0,-0x40(%ebp)
c14e87dc:	74 5e                	je     c14e883c <generic_make_request+0x302>
c14e87de:	8b 50 08             	mov    0x8(%eax),%edx
c14e87e1:	8b 82 84 00 00 00    	mov    0x84(%edx),%eax
c14e87e7:	a8 01                	test   $0x1,%al
c14e87e9:	74 04                	je     c14e87ef <generic_make_request+0x2b5>
c14e87eb:	f3 90                	pause  
c14e87ed:	eb f2                	jmp    c14e87e1 <generic_make_request+0x2a7>
c14e87ef:	8b 72 7c             	mov    0x7c(%edx),%esi
c14e87f2:	8b ba 80 00 00 00    	mov    0x80(%edx),%edi
c14e87f8:	39 82 84 00 00 00    	cmp    %eax,0x84(%edx)
c14e87fe:	75 e1                	jne    c14e87e1 <generic_make_request+0x2a7>
c14e8800:	89 f0                	mov    %esi,%eax
c14e8802:	89 fa                	mov    %edi,%edx
c14e8804:	0f ac d0 09          	shrd   $0x9,%edx,%eax
c14e8808:	c1 fa 09             	sar    $0x9,%edx
c14e880b:	89 d1                	mov    %edx,%ecx
c14e880d:	09 c1                	or     %eax,%ecx
c14e880f:	74 2b                	je     c14e883c <generic_make_request+0x302>
c14e8811:	8b 33                	mov    (%ebx),%esi
c14e8813:	8b 4b 04             	mov    0x4(%ebx),%ecx
c14e8816:	83 fa 00             	cmp    $0x0,%edx
c14e8819:	77 05                	ja     c14e8820 <generic_make_request+0x2e6>
c14e881b:	3b 45 c0             	cmp    -0x40(%ebp),%eax
c14e881e:	72 10                	jb     c14e8830 <generic_make_request+0x2f6>
c14e8820:	2b 45 9c             	sub    -0x64(%ebp),%eax
c14e8823:	1b 55 a0             	sbb    -0x60(%ebp),%edx
c14e8826:	39 ca                	cmp    %ecx,%edx
c14e8828:	77 12                	ja     c14e883c <generic_make_request+0x302>
c14e882a:	72 04                	jb     c14e8830 <generic_make_request+0x2f6>
c14e882c:	39 f0                	cmp    %esi,%eax
c14e882e:	73 0c                	jae    c14e883c <generic_make_request+0x302>
c14e8830:	89 d8                	mov    %ebx,%eax
c14e8832:	e8 87 e7 ff ff       	call   c14e6fbe <handle_bad_sector>
c14e8837:	e9 4d fe ff ff       	jmp    c14e8689 <generic_make_request+0x14f>
c14e883c:	8b 43 14             	mov    0x14(%ebx),%eax
c14e883f:	a9 00 10 80 00       	test   $0x801000,%eax
c14e8844:	74 1a                	je     c14e8860 <generic_make_request+0x326>
c14e8846:	8b 55 c4             	mov    -0x3c(%ebp),%edx
c14e8849:	83 ba 7c 02 00 00 00 	cmpl   $0x0,0x27c(%edx)
c14e8850:	75 0e                	jne    c14e8860 <generic_make_request+0x326>
c14e8852:	25 ff ef 7f ff       	and    $0xff7fefff,%eax
c14e8857:	89 43 14             	mov    %eax,0x14(%ebx)
c14e885a:	83 7d c0 00          	cmpl   $0x0,-0x40(%ebp)
c14e885e:	74 6d                	je     c14e88cd <generic_make_request+0x393>
c14e8860:	8b 43 14             	mov    0x14(%ebx),%eax
c14e8863:	a8 40                	test   $0x40,%al
c14e8865:	74 2d                	je     c14e8894 <generic_make_request+0x35a>
c14e8867:	8b 4d c4             	mov    -0x3c(%ebp),%ecx
c14e886a:	8b 91 a4 01 00 00    	mov    0x1a4(%ecx),%edx
c14e8870:	80 e6 40             	and    $0x40,%dh
c14e8873:	74 5c                	je     c14e88d1 <generic_make_request+0x397>
c14e8875:	a9 00 00 00 08       	test   $0x8000000,%eax
c14e887a:	74 18                	je     c14e8894 <generic_make_request+0x35a>
c14e887c:	8b 81 a4 01 00 00    	mov    0x1a4(%ecx),%eax
c14e8882:	f6 c4 40             	test   $0x40,%ah
c14e8885:	74 4a                	je     c14e88d1 <generic_make_request+0x397>
c14e8887:	8b 81 a4 01 00 00    	mov    0x1a4(%ecx),%eax
c14e888d:	a9 00 00 02 00       	test   $0x20000,%eax
c14e8892:	74 3d                	je     c14e88d1 <generic_make_request+0x397>
c14e8894:	85 db                	test   %ebx,%ebx
c14e8896:	74 45                	je     c14e88dd <generic_make_request+0x3a3>
c14e8898:	e9 00 00 00 00       	jmp    c14e889d <generic_make_request+0x363>
c14e889d:	eb 1c                	jmp    c14e88bb <generic_make_request+0x381>
c14e889f:	8b 35 fc 4e 2a c2    	mov    0xc22a4efc,%esi
c14e88a5:	85 f6                	test   %esi,%esi
c14e88a7:	74 12                	je     c14e88bb <generic_make_request+0x381>
c14e88a9:	8b 46 04             	mov    0x4(%esi),%eax
c14e88ac:	89 d9                	mov    %ebx,%ecx
c14e88ae:	8b 55 c4             	mov    -0x3c(%ebp),%edx
c14e88b1:	ff 16                	call   *(%esi)
c14e88b3:	83 c6 08             	add    $0x8,%esi
c14e88b6:	83 3e 00             	cmpl   $0x0,(%esi)
c14e88b9:	eb ec                	jmp    c14e88a7 <generic_make_request+0x36d>
c14e88bb:	89 da                	mov    %ebx,%edx
c14e88bd:	8b 45 c4             	mov    -0x3c(%ebp),%eax
c14e88c0:	ff 50 44             	call   *0x44(%eax)
c14e88c3:	85 c0                	test   %eax,%eax
c14e88c5:	0f 85 7d fd ff ff    	jne    c14e8648 <generic_make_request+0x10e>
c14e88cb:	eb 10                	jmp    c14e88dd <generic_make_request+0x3a3>
c14e88cd:	31 d2                	xor    %edx,%edx
c14e88cf:	eb 05                	jmp    c14e88d6 <generic_make_request+0x39c>
c14e88d1:	ba a1 ff ff ff       	mov    $0xffffffa1,%edx
c14e88d6:	89 d8                	mov    %ebx,%eax
c14e88d8:	e8 b8 49 c0 ff       	call   c10ed295 <bio_endio>
c14e88dd:	8b 55 b0             	mov    -0x50(%ebp),%edx
c14e88e0:	8b 82 f4 02 00 00    	mov    0x2f4(%edx),%eax
c14e88e6:	8b 18                	mov    (%eax),%ebx
c14e88e8:	85 db                	test   %ebx,%ebx
c14e88ea:	74 1c                	je     c14e8908 <generic_make_request+0x3ce>
c14e88ec:	8b 53 08             	mov    0x8(%ebx),%edx
c14e88ef:	89 10                	mov    %edx,(%eax)
c14e88f1:	85 d2                	test   %edx,%edx
c14e88f3:	75 07                	jne    c14e88fc <generic_make_request+0x3c2>
c14e88f5:	c7 40 04 00 00 00 00 	movl   $0x0,0x4(%eax)
c14e88fc:	c7 43 08 00 00 00 00 	movl   $0x0,0x8(%ebx)
c14e8903:	e9 a1 fc ff ff       	jmp    c14e85a9 <generic_make_request+0x6f>
c14e8908:	8b 4d b0             	mov    -0x50(%ebp),%ecx
c14e890b:	c7 81 f4 02 00 00 00 	movl   $0x0,0x2f4(%ecx)
c14e8912:	00 00 00 
c14e8915:	8b 45 f0             	mov    -0x10(%ebp),%eax
c14e8918:	65 33 05 14 00 00 00 	xor    %gs:0x14,%eax
c14e891f:	74 05                	je     c14e8926 <generic_make_request+0x3ec>
c14e8921:	e8 56 27 b5 ff       	call   c103b07c <__stack_chk_fail>
c14e8926:	83 c4 6c             	add    $0x6c,%esp
c14e8929:	5b                   	pop    %ebx
c14e892a:	5e                   	pop    %esi
c14e892b:	5f                   	pop    %edi
c14e892c:	5d                   	pop    %ebp

c14e892d: c3 ret

The crash is at:

  c14e85ac:       c1 ea 09                shr    $0x9,%edx
  c14e85af:       89 55 c0                mov    %edx,-0x40(%ebp)
  c14e85b2:       e8 e3 8c 88 00          call   c1d7129a <_cond_resched>
  c14e85b7:       83 7d c0 00             cmpl   $0x0,-0x40(%ebp)
  c14e85bb:       74 69                   je     c14e8626 <generic_make_request+0xec>
  c14e85bd:       8b 43 0c                mov    0xc(%ebx),%eax
* c14e85c0:       8b 40 08                mov    0x8(%eax),%eax   <==== [ **CRASH** ]
  c14e85c3:       8b 90 84 00 00 00       mov    0x84(%eax),%edx
  c14e85c9:       f6 c2 01                test   $0x1,%dl

which corresponds to:

1414
1415 if (!nr_sectors)
1416 return 0;
1417
1418 /* Test device or partition size, when known. */
1419 maxsector = i_size_read(bio->bi_bdev->bd_inode) >> 9; <==== [ **CRASH** ]
1420 if (maxsector) {
1421 sector_t sector = bio->bi_sector;
1422
1423 if (maxsector < nr_sectors || maxsector - nr_sectors < sector) {

bio->bi_bdev has become NULL?

I do not think the _cond_resched() was called, judging from stack contents. But we just had an IRQ:

[<c1d74030>] ? common_interrupt+0x30/0x40

So we might have raced with block IO IRQ queue-completion/submission activites.

But maybe it was a reschedule after all, just the stack does not carry any traces of it anymore. IRQs do not clear ->bi_bdev, right? Unless the bio refcounts are wrong and an IRQ's completion actually frees the bio, right?

I've built a CONFIG_DEBUG_PAGEALLOC=y and CONFIG_SLUB_DEBUG=y kernel, maybe the crash triggers in a more revealing way.

Thanks,

Ingo

#
# Automatically generated make config: don't edit
# Linux/i386 2.6.39-rc5 Kernel Configuration
# Wed May 4 11:12:22 2011
#
# CONFIG_64BIT is not set
CONFIG_X86_32=y
# CONFIG_X86_64 is not set
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y CONFIG_OUTPUT_FORMAT="elf32-i386" CONFIG_ARCH_DEFCONFIG="arch/x86/configs/i386_defconfig" CONFIG_GENERIC_CMOS_UPDATE=y CONFIG_CLOCKSOURCE_WATCHDOG=y CONFIG_GENERIC_CLOCKEVENTS=y CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y CONFIG_HAVE_LATENCYTOP_SUPPORT=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_NEED_DMA_MAP_STATE=y CONFIG_NEED_SG_DMA_LENGTH=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_GENERIC_GPIO=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set CONFIG_RWSEM_XCHGADD_ALGORITHM=y CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y CONFIG_GENERIC_CALIBRATE_DELAY=y
# CONFIG_GENERIC_TIME_VSYSCALL is not set CONFIG_ARCH_HAS_CPU_RELAX=y CONFIG_ARCH_HAS_DEFAULT_IDLE=y CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y CONFIG_HAVE_SETUP_PER_CPU_AREA=y CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
# CONFIG_HAVE_CPUMASK_OF_CPU_MAP is not set CONFIG_ARCH_HIBERNATION_POSSIBLE=y CONFIG_ARCH_SUSPEND_POSSIBLE=y
# CONFIG_ZONE_DMA32 is not set CONFIG_ARCH_POPULATES_NODE_MAP=y
# CONFIG_AUDIT_ARCH is not set CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y CONFIG_HAVE_INTEL_TXT=y
CONFIG_X86_32_SMP=y
CONFIG_X86_HT=y CONFIG_ARCH_HWEIGHT_CFLAGS="-fcall-saved-ecx -fcall-saved-edx" CONFIG_KTIME_SCALAR=y CONFIG_ARCH_CPU_PROBE_RELEASE=y CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config" CONFIG_CONSTRUCTORS=y
CONFIG_HAVE_IRQ_WORK=y
CONFIG_IRQ_WORK=y

#
# General setup
#
CONFIG_EXPERIMENTAL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_CROSS_COMPILE=""
CONFIG_LOCALVERSION="-i486-1sys"
# CONFIG_LOCALVERSION_AUTO is not set
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_KERNEL_GZIP=y
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
# CONFIG_KERNEL_XZ is not set
# CONFIG_KERNEL_LZO is not set
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_BSD_PROCESS_ACCT_V3=y
# CONFIG_FHANDLE is not set
# CONFIG_TASKSTATS is not set
CONFIG_AUDIT=y
CONFIG_AUDITSYSCALL=y
CONFIG_AUDIT_WATCH=y
CONFIG_AUDIT_TREE=y
CONFIG_HAVE_GENERIC_HARDIRQS=y

#
# IRQ subsystem
#
CONFIG_GENERIC_HARDIRQS=y
CONFIG_HAVE_SPARSE_IRQ=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_IRQ_SHOW=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_IRQ_FORCED_THREADING=y
# CONFIG_SPARSE_IRQ is not set

#
# RCU Subsystem
#
CONFIG_TREE_RCU=y
# CONFIG_PREEMPT_RCU is not set
# CONFIG_RCU_TRACE is not set
CONFIG_RCU_FANOUT=32
# CONFIG_RCU_FANOUT_EXACT is not set
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=21
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
CONFIG_CGROUPS=y
# CONFIG_CGROUP_DEBUG is not set
# CONFIG_CGROUP_NS is not set
# CONFIG_CGROUP_FREEZER is not set
# CONFIG_CGROUP_DEVICE is not set
# CONFIG_CPUSETS is not set
# CONFIG_CGROUP_CPUACCT is not set
# CONFIG_RESOURCE_COUNTERS is not set
# CONFIG_CGROUP_PERF is not set
CONFIG_CGROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
# CONFIG_RT_GROUP_SCHED is not set
# CONFIG_BLK_CGROUP is not set
CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
CONFIG_SCHED_AUTOGROUP=y
CONFIG_SYSFS_DEPRECATED=y
# CONFIG_SYSFS_DEPRECATED_V2 is not set
CONFIG_RELAY=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_SYSCTL=y
CONFIG_ANON_INODES=y
CONFIG_EXPERT=y
CONFIG_UID16=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
CONFIG_EMBEDDED=y
CONFIG_HAVE_PERF_EVENTS=y

#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
# CONFIG_PERF_COUNTERS is not set
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_PCI_QUIRKS=y
CONFIG_SLUB_DEBUG=y
CONFIG_COMPAT_BRK=y
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLOB is not set
CONFIG_PROFILING=y
CONFIG_TRACEPOINTS=y
CONFIG_OPROFILE=y
CONFIG_OPROFILE_EVENT_MULTIPLEX=y
CONFIG_HAVE_OPROFILE=y
# CONFIG_KPROBES is not set
CONFIG_JUMP_LABEL=y
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_ATTRS=y
CONFIG_USE_GENERIC_SMP_HELPERS=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_DMA_API_DEBUG=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
CONFIG_HAVE_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_PERF_EVENTS_NMI=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y

#
# GCOV-based kernel profiling
#
# CONFIG_GCOV_KERNEL is not set
CONFIG_HAVE_GENERIC_DMA_COHERENT=y
CONFIG_SLABINFO=y
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
CONFIG_MODULES=y
CONFIG_MODULE_FORCE_LOAD=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
# CONFIG_MODVERSIONS is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
CONFIG_LBDAF=y
CONFIG_BLK_DEV_BSG=y
CONFIG_BLK_DEV_INTEGRITY=y

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y CONFIG_IOSCHED_DEADLINE=y CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_DEADLINE is not set CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set CONFIG_DEFAULT_IOSCHED="cfq" CONFIG_PREEMPT_NOTIFIERS=y
CONFIG_PADATA=y
# CONFIG_INLINE_SPIN_TRYLOCK is not set
# CONFIG_INLINE_SPIN_TRYLOCK_BH is not set # CONFIG_INLINE_SPIN_LOCK is not set
# CONFIG_INLINE_SPIN_LOCK_BH is not set
# CONFIG_INLINE_SPIN_LOCK_IRQ is not set
# CONFIG_INLINE_SPIN_LOCK_IRQSAVE is not set CONFIG_INLINE_SPIN_UNLOCK=y
# CONFIG_INLINE_SPIN_UNLOCK_BH is not set CONFIG_INLINE_SPIN_UNLOCK_IRQ=y
# CONFIG_INLINE_SPIN_UNLOCK_IRQRESTORE is not set # CONFIG_INLINE_READ_TRYLOCK is not set
# CONFIG_INLINE_READ_LOCK is not set
# CONFIG_INLINE_READ_LOCK_BH is not set
# CONFIG_INLINE_READ_LOCK_IRQ is not set
# CONFIG_INLINE_READ_LOCK_IRQSAVE is not set CONFIG_INLINE_READ_UNLOCK=y
# CONFIG_INLINE_READ_UNLOCK_BH is not set CONFIG_INLINE_READ_UNLOCK_IRQ=y
# CONFIG_INLINE_READ_UNLOCK_IRQRESTORE is not set # CONFIG_INLINE_WRITE_TRYLOCK is not set
# CONFIG_INLINE_WRITE_LOCK is not set
# CONFIG_INLINE_WRITE_LOCK_BH is not set
# CONFIG_INLINE_WRITE_LOCK_IRQ is not set # CONFIG_INLINE_WRITE_LOCK_IRQSAVE is not set CONFIG_INLINE_WRITE_UNLOCK=y
# CONFIG_INLINE_WRITE_UNLOCK_BH is not set CONFIG_INLINE_WRITE_UNLOCK_IRQ=y
# CONFIG_INLINE_WRITE_UNLOCK_IRQRESTORE is not set CONFIG_MUTEX_SPIN_ON_OWNER=y CONFIG_FREEZER=y

#
# Processor type and features
#
CONFIG_TICK_ONESHOT=y
# CONFIG_NO_HZ is not set
CONFIG_HIGH_RES_TIMERS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_SMP=y
CONFIG_X86_MPPARSE=y
CONFIG_X86_BIGSMP=y
CONFIG_X86_EXTENDED_PLATFORM=y
CONFIG_X86_ELAN=y
# CONFIG_X86_MRST is not set
CONFIG_X86_RDC321X=y
# CONFIG_X86_32_NON_STANDARD is not set
CONFIG_X86_SUPPORTS_MEMORY_FAILURE=y
CONFIG_X86_32_IRIS=y
CONFIG_SCHED_OMIT_FRAME_POINTER=y
CONFIG_PARAVIRT_GUEST=y
# CONFIG_XEN_PRIVILEGED_GUEST is not set
CONFIG_KVM_CLOCK=y
CONFIG_KVM_GUEST=y
CONFIG_LGUEST_GUEST=y
CONFIG_PARAVIRT=y
# CONFIG_PARAVIRT_SPINLOCKS is not set
CONFIG_PARAVIRT_CLOCK=y
CONFIG_NO_BOOTMEM=y
CONFIG_MEMTEST=y
CONFIG_X86_INTERNODE_CACHE_SHIFT=4
CONFIG_X86_CMPXCHG=y
CONFIG_CMPXCHG_LOCAL=y
CONFIG_X86_L1_CACHE_SHIFT=4
CONFIG_X86_XADD=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_ALIGNMENT_16=y
CONFIG_X86_MINIMUM_CPU_FAMILY=4
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_PROCESSOR_SELECT=y
CONFIG_CPU_SUP_INTEL=y
CONFIG_CPU_SUP_CYRIX_32=y
CONFIG_CPU_SUP_AMD=y
CONFIG_CPU_SUP_CENTAUR=y
CONFIG_CPU_SUP_TRANSMETA_32=y
CONFIG_CPU_SUP_UMC_32=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_DMI=y
# CONFIG_IOMMU_HELPER is not set
CONFIG_IOMMU_API=y
CONFIG_NR_CPUS=32
CONFIG_SCHED_SMT=y
CONFIG_SCHED_MC=y
# CONFIG_IRQ_TIME_ACCOUNTING is not set
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS=y
CONFIG_X86_MCE=y
CONFIG_X86_MCE_INTEL=y
CONFIG_X86_MCE_AMD=y
CONFIG_X86_ANCIENT_MCE=y
CONFIG_X86_MCE_THRESHOLD=y
CONFIG_X86_MCE_INJECT=y
CONFIG_X86_THERMAL_VECTOR=y
CONFIG_VM86=y
# CONFIG_TOSHIBA is not set
CONFIG_I8K=y
CONFIG_X86_REBOOTFIXUPS=y
CONFIG_MICROCODE=y
CONFIG_MICROCODE_INTEL=y
CONFIG_MICROCODE_AMD=y
CONFIG_MICROCODE_OLD_INTERFACE=y
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
# CONFIG_NOHIGHMEM is not set
CONFIG_HIGHMEM4G=y
# CONFIG_HIGHMEM64G is not set
CONFIG_VMSPLIT_3G=y
# CONFIG_VMSPLIT_3G_OPT is not set
# CONFIG_VMSPLIT_2G is not set
# CONFIG_VMSPLIT_2G_OPT is not set
# CONFIG_VMSPLIT_1G is not set
CONFIG_PAGE_OFFSET=0xC0000000
CONFIG_HIGHMEM=y
# CONFIG_ARCH_PHYS_ADDR_T_64BIT is not set
# CONFIG_ARCH_DMA_ADDR_T_64BIT is not set
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_ILLEGAL_POINTER_VALUE=0
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_FLATMEM_MANUAL=y
# CONFIG_SPARSEMEM_MANUAL is not set
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
CONFIG_SPARSEMEM_STATIC=y
CONFIG_HAVE_MEMBLOCK=y
CONFIG_PAGEFLAGS_EXTENDED=y
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_COMPACTION=y
CONFIG_MIGRATION=y
# CONFIG_PHYS_ADDR_T_64BIT is not set
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_MMU_NOTIFIER=y
CONFIG_KSM=y
CONFIG_DEFAULT_MMAP_MIN_ADDR=65536
CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y
CONFIG_MEMORY_FAILURE=y
CONFIG_TRANSPARENT_HUGEPAGE=y
CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS=y
# CONFIG_TRANSPARENT_HUGEPAGE_MADVISE is not set
CONFIG_HIGHPTE=y
CONFIG_X86_CHECK_BIOS_CORRUPTION=y
CONFIG_X86_BOOTPARAM_MEMORY_CORRUPTION_CHECK=y
CONFIG_X86_RESERVE_LOW=64
CONFIG_MATH_EMULATION=y
CONFIG_MTRR=y
CONFIG_MTRR_SANITIZER=y
CONFIG_MTRR_SANITIZER_ENABLE_DEFAULT=0
CONFIG_MTRR_SANITIZER_SPARE_REG_NR_DEFAULT=1
CONFIG_X86_PAT=y
CONFIG_ARCH_USES_PG_UNCACHED=y
CONFIG_EFI=y
CONFIG_SECCOMP=y
CONFIG_CC_STACKPROTECTOR=y
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_SCHED_HRTICK=y
# CONFIG_KEXEC is not set
# CONFIG_CRASH_DUMP is not set
CONFIG_PHYSICAL_START=0x1000000
# CONFIG_RELOCATABLE is not set
CONFIG_PHYSICAL_ALIGN=0x100000
CONFIG_HOTPLUG_CPU=y
CONFIG_COMPAT_VDSO=y
# CONFIG_CMDLINE_BOOL is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y

#
# Power management and ACPI options
#
CONFIG_SUSPEND=y
CONFIG_SUSPEND_FREEZER=y
CONFIG_HIBERNATE_CALLBACKS=y
CONFIG_HIBERNATION=y
CONFIG_PM_STD_PARTITION="\"\""
CONFIG_PM_SLEEP=y
CONFIG_PM_SLEEP_SMP=y
CONFIG_PM_RUNTIME=y
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
CONFIG_ACPI=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_PROCFS=y
CONFIG_ACPI_PROCFS_POWER=y
CONFIG_ACPI_POWER_METER=m
# CONFIG_ACPI_EC_DEBUGFS is not set
CONFIG_ACPI_PROC_EVENT=y
CONFIG_ACPI_AC=y
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_VIDEO=y
CONFIG_ACPI_FAN=y
CONFIG_ACPI_DOCK=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_IPMI=m
CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_PROCESSOR_AGGREGATOR=y
CONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_CUSTOM_DSDT is not set
CONFIG_ACPI_BLACKLIST_YEAR=2001
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_PCI_SLOT=y
CONFIG_X86_PM_TIMER=y
CONFIG_ACPI_CONTAINER=y
CONFIG_ACPI_SBS=y
CONFIG_ACPI_HED=y
CONFIG_ACPI_APEI=y
CONFIG_ACPI_APEI_GHES=m
# CONFIG_ACPI_APEI_PCIEAER is not set
# CONFIG_ACPI_APEI_EINJ is not set
# CONFIG_ACPI_APEI_ERST_DEBUG is not set
CONFIG_SFI=y
CONFIG_X86_APM_BOOT=y
CONFIG_APM=y
# CONFIG_APM_IGNORE_USER_SUSPEND is not set
# CONFIG_APM_DO_ENABLE is not set
# CONFIG_APM_CPU_IDLE is not set
# CONFIG_APM_DISPLAY_BLANK is not set
# CONFIG_APM_ALLOW_INTS is not set

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=y
# CONFIG_CPU_FREQ_DEBUG is not set CONFIG_CPU_FREQ_STAT=y CONFIG_CPU_FREQ_STAT_DETAILS=y CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set # CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set # CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set CONFIG_CPU_FREQ_GOV_PERFORMANCE=y CONFIG_CPU_FREQ_GOV_POWERSAVE=y CONFIG_CPU_FREQ_GOV_USERSPACE=y CONFIG_CPU_FREQ_GOV_ONDEMAND=y CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y

#
# CPUFreq processor drivers
#
CONFIG_X86_PCC_CPUFREQ=y
CONFIG_X86_ACPI_CPUFREQ=y
CONFIG_ELAN_CPUFREQ=y
CONFIG_SC520_CPUFREQ=y
CONFIG_X86_POWERNOW_K6=y
CONFIG_X86_POWERNOW_K7=y
CONFIG_X86_POWERNOW_K7_ACPI=y
CONFIG_X86_POWERNOW_K8=y
CONFIG_X86_GX_SUSPMOD=y
CONFIG_X86_SPEEDSTEP_CENTRINO=y
CONFIG_X86_SPEEDSTEP_CENTRINO_TABLE=y
CONFIG_X86_SPEEDSTEP_ICH=y
CONFIG_X86_SPEEDSTEP_SMI=y
CONFIG_X86_P4_CLOCKMOD=y
CONFIG_X86_CPUFREQ_NFORCE2=y
CONFIG_X86_LONGRUN=y
CONFIG_X86_LONGHAUL=y
CONFIG_X86_E_POWERSAVER=y

# shared options
#
CONFIG_X86_SPEEDSTEP_LIB=y
CONFIG_X86_SPEEDSTEP_RELAXED_CAP_CHECK=y
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_INTEL_IDLE=y

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
# CONFIG_PCI_GOOLPC is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_OLPC=y
CONFIG_PCI_DOMAINS=y
CONFIG_PCI_CNB20LE_QUIRK=y
CONFIG_DMAR=y
CONFIG_DMAR_DEFAULT_ON=y
CONFIG_DMAR_FLOPPY_WA=y
CONFIG_PCIEPORTBUS=y
CONFIG_HOTPLUG_PCI_PCIE=y
CONFIG_PCIEAER=y
CONFIG_PCIE_ECRC=y
CONFIG_PCIEAER_INJECT=y
CONFIG_PCIEASPM=y
CONFIG_PCIEASPM_DEBUG=y
CONFIG_PCIE_PME=y
CONFIG_ARCH_SUPPORTS_MSI=y
CONFIG_PCI_MSI=y
# CONFIG_PCI_STUB is not set
CONFIG_HT_IRQ=y
CONFIG_PCI_IOV=y
CONFIG_PCI_IOAPIC=y
CONFIG_PCI_LABEL=y
CONFIG_ISA_DMA_API=y
CONFIG_ISA=y
CONFIG_EISA=y
CONFIG_EISA_VLB_PRIMING=y
CONFIG_EISA_PCI_EISA=y
CONFIG_EISA_VIRTUAL_ROOT=y
CONFIG_EISA_NAMES=y
CONFIG_MCA=y
CONFIG_MCA_LEGACY=y
CONFIG_MCA_PROC_FS=y
CONFIG_SCx200=y
CONFIG_SCx200HR_TIMER=y
CONFIG_OLPC=y
CONFIG_OLPC_XO1=m
CONFIG_AMD_NB=y
CONFIG_PCCARD=y
CONFIG_PCMCIA=y
CONFIG_PCMCIA_LOAD_CIS=y
CONFIG_CARDBUS=y

#
# PC-card bridges
#
CONFIG_YENTA=y
CONFIG_YENTA_O2=y
CONFIG_YENTA_RICOH=y
CONFIG_YENTA_TI=y
CONFIG_YENTA_ENE_TUNE=y
CONFIG_YENTA_TOSHIBA=y
CONFIG_PD6729=y
CONFIG_I82092=y
CONFIG_I82365=y
CONFIG_TCIC=y
CONFIG_PCMCIA_PROBE=y
CONFIG_PCCARD_NONSTATIC=y
CONFIG_HOTPLUG_PCI=y
# CONFIG_HOTPLUG_PCI_FAKE is not set
CONFIG_HOTPLUG_PCI_COMPAQ=y
# CONFIG_HOTPLUG_PCI_COMPAQ_NVRAM is not set
CONFIG_HOTPLUG_PCI_IBM=y
CONFIG_HOTPLUG_PCI_ACPI=y
CONFIG_HOTPLUG_PCI_ACPI_IBM=y
CONFIG_HOTPLUG_PCI_CPCI=y
CONFIG_HOTPLUG_PCI_CPCI_ZT5550=y
CONFIG_HOTPLUG_PCI_CPCI_GENERIC=y
CONFIG_HOTPLUG_PCI_SHPC=y
CONFIG_RAPIDIO=y
CONFIG_RAPIDIO_DISC_TIMEOUT=30
CONFIG_RAPIDIO_ENABLE_RX_TX_PORTS=y
CONFIG_RAPIDIO_TSI57X=y
CONFIG_RAPIDIO_CPS_XX=y
CONFIG_RAPIDIO_TSI568=y
CONFIG_RAPIDIO_CPS_GEN2=y
CONFIG_RAPIDIO_TSI500=y
CONFIG_RAPIDIO_DEBUG=y

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
CONFIG_HAVE_AOUT=y
CONFIG_BINFMT_AOUT=y
CONFIG_BINFMT_MISC=y
CONFIG_HAVE_ATOMIC_IOMAP=y
CONFIG_HAVE_TEXT_POKE_SMP=y
CONFIG_NET=y

#
# Networking options
#
CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_XFRM=y
CONFIG_XFRM_USER=y
CONFIG_XFRM_SUB_POLICY=y
CONFIG_XFRM_MIGRATE=y
CONFIG_XFRM_STATISTICS=y
CONFIG_XFRM_IPCOMP=m
CONFIG_NET_KEY=m
CONFIG_NET_KEY_MIGRATE=y
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_FIB_TRIE_STATS=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_IP_ROUTE_VERBOSE=y
CONFIG_IP_ROUTE_CLASSID=y
CONFIG_IP_PNP=y
CONFIG_IP_PNP_DHCP=y
CONFIG_IP_PNP_BOOTP=y
CONFIG_IP_PNP_RARP=y
CONFIG_NET_IPIP=y
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
CONFIG_NET_IPGRE_BROADCAST=y
CONFIG_IP_MROUTE=y
CONFIG_IP_MROUTE_MULTIPLE_TABLES=y
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y
CONFIG_ARPD=y
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_IPCOMP=m
CONFIG_INET_XFRM_TUNNEL=m
CONFIG_INET_TUNNEL=y
CONFIG_INET_XFRM_MODE_TRANSPORT=m
CONFIG_INET_XFRM_MODE_TUNNEL=m
CONFIG_INET_XFRM_MODE_BEET=m
CONFIG_INET_LRO=y
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
CONFIG_TCP_CONG_ADVANCED=y
CONFIG_TCP_CONG_BIC=m
CONFIG_TCP_CONG_CUBIC=y
CONFIG_TCP_CONG_WESTWOOD=m
CONFIG_TCP_CONG_HTCP=m
CONFIG_TCP_CONG_HSTCP=m
CONFIG_TCP_CONG_HYBLA=m
CONFIG_TCP_CONG_VEGAS=m
CONFIG_TCP_CONG_SCALABLE=m
CONFIG_TCP_CONG_LP=m
CONFIG_TCP_CONG_VENO=m
CONFIG_TCP_CONG_YEAH=m
CONFIG_TCP_CONG_ILLINOIS=m
CONFIG_DEFAULT_CUBIC=y
# CONFIG_DEFAULT_RENO is not set
CONFIG_DEFAULT_TCP_CONG="cubic"
CONFIG_TCP_MD5SIG=y
CONFIG_IPV6=y
CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
CONFIG_IPV6_OPTIMISTIC_DAD=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
CONFIG_IPV6_MIP6=m
CONFIG_INET6_XFRM_TUNNEL=m
CONFIG_INET6_TUNNEL=m
CONFIG_INET6_XFRM_MODE_TRANSPORT=m
CONFIG_INET6_XFRM_MODE_TUNNEL=m
CONFIG_INET6_XFRM_MODE_BEET=m
CONFIG_INET6_XFRM_MODE_ROUTEOPTIMIZATION=m
CONFIG_IPV6_SIT=m
CONFIG_IPV6_SIT_6RD=y
CONFIG_IPV6_NDISC_NODETYPE=y
CONFIG_IPV6_TUNNEL=m
CONFIG_IPV6_MULTIPLE_TABLES=y
CONFIG_IPV6_SUBTREES=y
CONFIG_IPV6_MROUTE=y
CONFIG_IPV6_MROUTE_MULTIPLE_TABLES=y
CONFIG_IPV6_PIMSM_V2=y
CONFIG_NETLABEL=y
# CONFIG_NETWORK_SECMARK is not set
# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
CONFIG_NETFILTER_ADVANCED=y
CONFIG_BRIDGE_NETFILTER=y

#
# Core Netfilter Configuration
#
CONFIG_NETFILTER_NETLINK=m
CONFIG_NETFILTER_NETLINK_QUEUE=m
CONFIG_NETFILTER_NETLINK_LOG=m
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_MARK=y
# CONFIG_NF_CONNTRACK_ZONES is not set
# CONFIG_NF_CONNTRACK_EVENTS is not set
# CONFIG_NF_CONNTRACK_TIMESTAMP is not set
CONFIG_NF_CT_PROTO_DCCP=m
CONFIG_NF_CT_PROTO_GRE=m
CONFIG_NF_CT_PROTO_SCTP=m
CONFIG_NF_CT_PROTO_UDPLITE=m
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_H323=m
CONFIG_NF_CONNTRACK_IRC=m
CONFIG_NF_CONNTRACK_BROADCAST=m
CONFIG_NF_CONNTRACK_NETBIOS_NS=m
CONFIG_NF_CONNTRACK_SNMP=m
CONFIG_NF_CONNTRACK_PPTP=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
CONFIG_NF_CT_NETLINK=m
CONFIG_NETFILTER_TPROXY=m
CONFIG_NETFILTER_XTABLES=m

#
# Xtables combined modules
#
CONFIG_NETFILTER_XT_MARK=m
CONFIG_NETFILTER_XT_CONNMARK=m
# CONFIG_NETFILTER_XT_SET is not set

#
# Xtables targets
#
CONFIG_NETFILTER_XT_TARGET_AUDIT=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_CONNMARK=m
CONFIG_NETFILTER_XT_TARGET_CT=m
CONFIG_NETFILTER_XT_TARGET_DSCP=m
CONFIG_NETFILTER_XT_TARGET_HL=m
CONFIG_NETFILTER_XT_TARGET_IDLETIMER=m
CONFIG_NETFILTER_XT_TARGET_LED=m
CONFIG_NETFILTER_XT_TARGET_MARK=m
CONFIG_NETFILTER_XT_TARGET_NFLOG=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_RATEEST=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m

#
# Xtables matches
#
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=m
CONFIG_NETFILTER_XT_MATCH_CLUSTER=m
CONFIG_NETFILTER_XT_MATCH_COMMENT=m
CONFIG_NETFILTER_XT_MATCH_CONNBYTES=m
CONFIG_NETFILTER_XT_MATCH_CONNLIMIT=m
CONFIG_NETFILTER_XT_MATCH_CONNMARK=m
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
CONFIG_NETFILTER_XT_MATCH_CPU=m
CONFIG_NETFILTER_XT_MATCH_DCCP=m
CONFIG_NETFILTER_XT_MATCH_DEVGROUP=m
CONFIG_NETFILTER_XT_MATCH_DSCP=m
CONFIG_NETFILTER_XT_MATCH_ESP=m
CONFIG_NETFILTER_XT_MATCH_HASHLIMIT=m
CONFIG_NETFILTER_XT_MATCH_HELPER=m
CONFIG_NETFILTER_XT_MATCH_HL=m
CONFIG_NETFILTER_XT_MATCH_IPRANGE=m
CONFIG_NETFILTER_XT_MATCH_IPVS=m
CONFIG_NETFILTER_XT_MATCH_LENGTH=m
CONFIG_NETFILTER_XT_MATCH_LIMIT=m
CONFIG_NETFILTER_XT_MATCH_MAC=m
CONFIG_NETFILTER_XT_MATCH_MARK=m
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=m
CONFIG_NETFILTER_XT_MATCH_OSF=m
CONFIG_NETFILTER_XT_MATCH_OWNER=m
CONFIG_NETFILTER_XT_MATCH_POLICY=m
CONFIG_NETFILTER_XT_MATCH_PHYSDEV=m
CONFIG_NETFILTER_XT_MATCH_PKTTYPE=m
CONFIG_NETFILTER_XT_MATCH_QUOTA=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
CONFIG_NETFILTER_XT_MATCH_SCTP=m
CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_NETFILTER_XT_MATCH_TCPMSS=m
CONFIG_NETFILTER_XT_MATCH_TIME=m
CONFIG_NETFILTER_XT_MATCH_U32=m
CONFIG_IP_SET=m
CONFIG_IP_SET_MAX=256
CONFIG_IP_SET_BITMAP_IP=m
CONFIG_IP_SET_BITMAP_IPMAC=m
CONFIG_IP_SET_BITMAP_PORT=m
CONFIG_IP_SET_HASH_IP=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
CONFIG_IP_SET_HASH_NET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_IP_VS=m
CONFIG_IP_VS_IPV6=y
# CONFIG_IP_VS_DEBUG is not set
CONFIG_IP_VS_TAB_BITS=12

#
# IPVS transport protocol load balancing support
#
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_PROTO_AH_ESP=y
CONFIG_IP_VS_PROTO_ESP=y
CONFIG_IP_VS_PROTO_AH=y
CONFIG_IP_VS_PROTO_SCTP=y

#
# IPVS scheduler
#
CONFIG_IP_VS_RR=m
CONFIG_IP_VS_WRR=m
CONFIG_IP_VS_LC=m
CONFIG_IP_VS_WLC=m
CONFIG_IP_VS_LBLC=m
CONFIG_IP_VS_LBLCR=m
CONFIG_IP_VS_DH=m
CONFIG_IP_VS_SH=m
CONFIG_IP_VS_SED=m
CONFIG_IP_VS_NQ=m

#
# IPVS application helper
#
CONFIG_IP_VS_FTP=m
CONFIG_IP_VS_NFCT=y
CONFIG_IP_VS_PE_SIP=m

#
# IP: Netfilter Configuration
#
CONFIG_NF_DEFRAG_IPV4=m
CONFIG_NF_CONNTRACK_IPV4=m
CONFIG_NF_CONNTRACK_PROC_COMPAT=y
CONFIG_IP_NF_QUEUE=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_TARGET_LOG=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT=m
CONFIG_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_NF_NAT_SNMP_BASIC=m
CONFIG_NF_NAT_PROTO_DCCP=m
CONFIG_NF_NAT_PROTO_GRE=m
CONFIG_NF_NAT_PROTO_UDPLITE=m
CONFIG_NF_NAT_PROTO_SCTP=m
CONFIG_NF_NAT_FTP=m
CONFIG_NF_NAT_IRC=m
CONFIG_NF_NAT_TFTP=m
CONFIG_NF_NAT_AMANDA=m
CONFIG_NF_NAT_PPTP=m
CONFIG_NF_NAT_H323=m
CONFIG_NF_NAT_SIP=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_CLUSTERIP=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_TTL=m
CONFIG_IP_NF_RAW=m
CONFIG_IP_NF_SECURITY=m
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m

#
# IPv6: Netfilter Configuration
#
CONFIG_NF_DEFRAG_IPV6=m
CONFIG_NF_CONNTRACK_IPV6=m
CONFIG_IP6_NF_QUEUE=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_MATCH_FRAG=m
CONFIG_IP6_NF_MATCH_OPTS=m
CONFIG_IP6_NF_MATCH_HL=m
CONFIG_IP6_NF_MATCH_IPV6HEADER=m
CONFIG_IP6_NF_MATCH_MH=m
CONFIG_IP6_NF_MATCH_RT=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_TARGET_LOG=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_IP6_NF_SECURITY=m

#
# DECnet: Netfilter Configuration
#
CONFIG_DECNET_NF_GRABULATOR=m
CONFIG_BRIDGE_NF_EBTABLES=m
CONFIG_BRIDGE_EBT_BROUTE=m
CONFIG_BRIDGE_EBT_T_FILTER=m
CONFIG_BRIDGE_EBT_T_NAT=m
CONFIG_BRIDGE_EBT_802_3=m
CONFIG_BRIDGE_EBT_AMONG=m
CONFIG_BRIDGE_EBT_ARP=m
CONFIG_BRIDGE_EBT_IP=m
CONFIG_BRIDGE_EBT_IP6=m
CONFIG_BRIDGE_EBT_LIMIT=m
CONFIG_BRIDGE_EBT_MARK=m
CONFIG_BRIDGE_EBT_PKTTYPE=m
CONFIG_BRIDGE_EBT_STP=m
CONFIG_BRIDGE_EBT_VLAN=m
CONFIG_BRIDGE_EBT_ARPREPLY=m
CONFIG_BRIDGE_EBT_DNAT=m
CONFIG_BRIDGE_EBT_MARK_T=m
CONFIG_BRIDGE_EBT_REDIRECT=m
CONFIG_BRIDGE_EBT_SNAT=m
CONFIG_BRIDGE_EBT_LOG=m
CONFIG_BRIDGE_EBT_ULOG=m
CONFIG_BRIDGE_EBT_NFLOG=m
CONFIG_IP_DCCP=m
CONFIG_INET_DCCP_DIAG=m

#
# DCCP CCIDs Configuration (EXPERIMENTAL)
#
# CONFIG_IP_DCCP_CCID2_DEBUG is not set
CONFIG_IP_DCCP_CCID3=y
# CONFIG_IP_DCCP_CCID3_DEBUG is not set
CONFIG_IP_DCCP_TFRC_LIB=y
CONFIG_IP_SCTP=y
# CONFIG_SCTP_DBG_MSG is not set
# CONFIG_SCTP_DBG_OBJCNT is not set
# CONFIG_SCTP_HMAC_NONE is not set
# CONFIG_SCTP_HMAC_SHA1 is not set
CONFIG_SCTP_HMAC_MD5=y
CONFIG_RDS=m
CONFIG_RDS_RDMA=m
CONFIG_RDS_TCP=m
# CONFIG_RDS_DEBUG is not set
CONFIG_TIPC=m
CONFIG_TIPC_ADVANCED=y
CONFIG_TIPC_PORTS=8191
CONFIG_TIPC_LOG=0
# CONFIG_TIPC_DEBUG is not set
CONFIG_ATM=y
CONFIG_ATM_CLIP=y
# CONFIG_ATM_CLIP_NO_ICMP is not set
CONFIG_ATM_LANE=y
CONFIG_ATM_MPOA=y
CONFIG_ATM_BR2684=m
# CONFIG_ATM_BR2684_IPFILTER is not set
CONFIG_L2TP=m
# CONFIG_L2TP_DEBUGFS is not set
CONFIG_L2TP_V3=y
CONFIG_L2TP_IP=m
CONFIG_L2TP_ETH=m
CONFIG_STP=m
CONFIG_GARP=m
CONFIG_BRIDGE=m
CONFIG_BRIDGE_IGMP_SNOOPING=y
CONFIG_NET_DSA=y
CONFIG_NET_DSA_TAG_DSA=y
CONFIG_NET_DSA_TAG_EDSA=y
CONFIG_NET_DSA_TAG_TRAILER=y
CONFIG_NET_DSA_MV88E6XXX=y
CONFIG_NET_DSA_MV88E6060=y
CONFIG_NET_DSA_MV88E6XXX_NEED_PPU=y
CONFIG_NET_DSA_MV88E6131=y
CONFIG_NET_DSA_MV88E6123_61_65=y
CONFIG_VLAN_8021Q=m
CONFIG_VLAN_8021Q_GVRP=y
CONFIG_DECNET=m
CONFIG_DECNET_ROUTER=y
CONFIG_LLC=y
CONFIG_LLC2=m
CONFIG_IPX=m
CONFIG_IPX_INTERN=y
CONFIG_ATALK=m
CONFIG_DEV_APPLETALK=m
CONFIG_LTPC=m
# CONFIG_COPS is not set
CONFIG_IPDDP=m
CONFIG_IPDDP_ENCAP=y
CONFIG_IPDDP_DECAP=y
CONFIG_X25=m
CONFIG_LAPB=m
CONFIG_ECONET=m
CONFIG_ECONET_AUNUDP=y
CONFIG_ECONET_NATIVE=y
CONFIG_WAN_ROUTER=y
CONFIG_PHONET=m
CONFIG_IEEE802154=m
CONFIG_NET_SCHED=y

#
# Queueing/Scheduling
#
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_HFSC=m
CONFIG_NET_SCH_ATM=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_MULTIQ=m
CONFIG_NET_SCH_RED=m
# CONFIG_NET_SCH_SFB is not set
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_DSMARK=m
CONFIG_NET_SCH_NETEM=m
CONFIG_NET_SCH_DRR=m
# CONFIG_NET_SCH_MQPRIO is not set
# CONFIG_NET_SCH_CHOKE is not set
CONFIG_NET_SCH_INGRESS=m

#
# Classification
#
CONFIG_NET_CLS=y
CONFIG_NET_CLS_BASIC=m
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
CONFIG_CLS_U32_PERF=y
CONFIG_CLS_U32_MARK=y
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
CONFIG_NET_CLS_FLOW=m
# CONFIG_NET_CLS_CGROUP is not set
# CONFIG_NET_EMATCH is not set
CONFIG_NET_CLS_ACT=y
CONFIG_NET_ACT_POLICE=y
CONFIG_NET_ACT_GACT=m
CONFIG_GACT_PROB=y
CONFIG_NET_ACT_MIRRED=m
CONFIG_NET_ACT_IPT=m
CONFIG_NET_ACT_NAT=m
CONFIG_NET_ACT_PEDIT=m
CONFIG_NET_ACT_SIMP=m
CONFIG_NET_ACT_SKBEDIT=m
CONFIG_NET_ACT_CSUM=m
CONFIG_NET_CLS_IND=y
CONFIG_NET_SCH_FIFO=y
CONFIG_DCB=y
CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
# CONFIG_BATMAN_ADV_DEBUG is not set
CONFIG_RPS=y
CONFIG_RFS_ACCEL=y
CONFIG_XPS=y

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_NET_DROP_MONITOR is not set
CONFIG_HAMRADIO=y

#
# Packet Radio protocols
#
CONFIG_AX25=m
CONFIG_AX25_DAMA_SLAVE=y
CONFIG_NETROM=m
CONFIG_ROSE=m

#
# AX.25 network device drivers
#
CONFIG_MKISS=m
CONFIG_6PACK=m
CONFIG_BPQETHER=m
CONFIG_SCC=m
CONFIG_SCC_DELAY=y
CONFIG_SCC_TRXECHO=y
CONFIG_BAYCOM_SER_FDX=m
CONFIG_BAYCOM_SER_HDX=m
CONFIG_BAYCOM_PAR=m
CONFIG_BAYCOM_EPP=m
CONFIG_YAM=m
CONFIG_CAN=m
CONFIG_CAN_RAW=m
CONFIG_CAN_BCM=m

#
# CAN Device Drivers
#
CONFIG_CAN_VCAN=m
CONFIG_CAN_SLCAN=m
CONFIG_CAN_DEV=m
CONFIG_CAN_CALC_BITTIMING=y
CONFIG_CAN_MCP251X=m
CONFIG_CAN_JANZ_ICAN3=m
CONFIG_PCH_CAN=m
CONFIG_CAN_SJA1000=m
CONFIG_CAN_SJA1000_ISA=m
CONFIG_CAN_SJA1000_PLATFORM=m
CONFIG_CAN_EMS_PCI=m
CONFIG_CAN_KVASER_PCI=m
CONFIG_CAN_PLX_PCI=m
CONFIG_CAN_TSCAN1=m
CONFIG_CAN_C_CAN=m
CONFIG_CAN_C_CAN_PLATFORM=m

#
# CAN USB interfaces
#
CONFIG_CAN_EMS_USB=m
CONFIG_CAN_ESD_USB2=m
CONFIG_CAN_SOFTING=m
CONFIG_CAN_SOFTING_CS=m
# CONFIG_CAN_DEBUG_DEVICES is not set
CONFIG_IRDA=m

#
# IrDA protocols
#
CONFIG_IRLAN=m
CONFIG_IRNET=m
CONFIG_IRCOMM=m
CONFIG_IRDA_ULTRA=y

#
# IrDA options
#
CONFIG_IRDA_CACHE_LAST_LSAP=y
CONFIG_IRDA_FAST_RR=y
# CONFIG_IRDA_DEBUG is not set

#
# Infrared-port device drivers
#

#
# SIR device drivers
#
CONFIG_IRTTY_SIR=m

#
# Dongle support
#
CONFIG_DONGLE=y
CONFIG_ESI_DONGLE=m
CONFIG_ACTISYS_DONGLE=m
CONFIG_TEKRAM_DONGLE=m
CONFIG_TOIM3232_DONGLE=m
CONFIG_LITELINK_DONGLE=m
CONFIG_MA600_DONGLE=m
CONFIG_GIRBIL_DONGLE=m
CONFIG_MCP2120_DONGLE=m
CONFIG_OLD_BELKIN_DONGLE=m
CONFIG_ACT200L_DONGLE=m
CONFIG_KINGSUN_DONGLE=m
CONFIG_KSDAZZLE_DONGLE=m
CONFIG_KS959_DONGLE=m

#
# FIR device drivers
#
CONFIG_USB_IRDA=m
CONFIG_SIGMATEL_FIR=m
CONFIG_NSC_FIR=m
CONFIG_WINBOND_FIR=m
CONFIG_TOSHIBA_FIR=m
CONFIG_SMC_IRCC_FIR=m
CONFIG_ALI_FIR=m
CONFIG_VLSI_FIR=m
CONFIG_VIA_FIR=m
CONFIG_MCS_FIR=m
CONFIG_BT=m
CONFIG_BT_L2CAP=y
CONFIG_BT_SCO=y
CONFIG_BT_RFCOMM=m
CONFIG_BT_RFCOMM_TTY=y
CONFIG_BT_BNEP=m
CONFIG_BT_BNEP_MC_FILTER=y
CONFIG_BT_BNEP_PROTO_FILTER=y
CONFIG_BT_CMTP=m
CONFIG_BT_HIDP=m

#
# Bluetooth device drivers
#
CONFIG_BT_HCIBTUSB=m
CONFIG_BT_HCIBTSDIO=m
CONFIG_BT_HCIUART=m
CONFIG_BT_HCIUART_H4=y
CONFIG_BT_HCIUART_BCSP=y
CONFIG_BT_HCIUART_ATH3K=y
CONFIG_BT_HCIUART_LL=y
CONFIG_BT_HCIBCM203X=m
CONFIG_BT_HCIBPA10X=m
CONFIG_BT_HCIBFUSB=m
CONFIG_BT_HCIDTL1=m
CONFIG_BT_HCIBT3C=m
CONFIG_BT_HCIBLUECARD=m
CONFIG_BT_HCIBTUART=m
CONFIG_BT_HCIVHCI=m
CONFIG_BT_MRVL=m
CONFIG_BT_MRVL_SDIO=m
CONFIG_BT_ATH3K=m
CONFIG_BT_WILINK=m
CONFIG_AF_RXRPC=y
# CONFIG_AF_RXRPC_DEBUG is not set
CONFIG_RXKAD=m
CONFIG_FIB_RULES=y
CONFIG_WIRELESS=y
CONFIG_WIRELESS_EXT=y
CONFIG_WEXT_CORE=y
CONFIG_WEXT_PROC=y
CONFIG_WEXT_SPY=y
CONFIG_WEXT_PRIV=y
CONFIG_CFG80211=m
# CONFIG_NL80211_TESTMODE is not set
# CONFIG_CFG80211_DEVELOPER_WARNINGS is not set
# CONFIG_CFG80211_REG_DEBUG is not set
CONFIG_CFG80211_DEFAULT_PS=y
# CONFIG_CFG80211_DEBUGFS is not set
# CONFIG_CFG80211_INTERNAL_REGDB is not set
CONFIG_CFG80211_WEXT=y
CONFIG_WIRELESS_EXT_SYSFS=y
CONFIG_LIB80211=y
CONFIG_LIB80211_CRYPT_WEP=y
CONFIG_LIB80211_CRYPT_CCMP=y
CONFIG_LIB80211_CRYPT_TKIP=y
# CONFIG_LIB80211_DEBUG is not set
CONFIG_MAC80211=m
CONFIG_MAC80211_HAS_RC=y
CONFIG_MAC80211_RC_PID=y
CONFIG_MAC80211_RC_MINSTREL=y
CONFIG_MAC80211_RC_MINSTREL_HT=y
# CONFIG_MAC80211_RC_DEFAULT_PID is not set
CONFIG_MAC80211_RC_DEFAULT_MINSTREL=y
CONFIG_MAC80211_RC_DEFAULT="minstrel_ht"
CONFIG_MAC80211_MESH=y
CONFIG_MAC80211_LEDS=y
# CONFIG_MAC80211_DEBUGFS is not set
# CONFIG_MAC80211_DEBUG_MENU is not set
CONFIG_WIMAX=m
CONFIG_WIMAX_DEBUG_LEVEL=8
CONFIG_RFKILL=y
CONFIG_RFKILL_LEDS=y
CONFIG_RFKILL_INPUT=y
CONFIG_NET_9P=m
CONFIG_NET_9P_VIRTIO=m
CONFIG_NET_9P_RDMA=m
# CONFIG_NET_9P_DEBUG is not set
CONFIG_CAIF=m
# CONFIG_CAIF_DEBUG is not set
CONFIG_CAIF_NETDEV=m
CONFIG_CEPH_LIB=y
# CONFIG_CEPH_LIB_PRETTYDEBUG is not set

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
CONFIG_EXTRA_FIRMWARE=""
# CONFIG_SYS_HYPERVISOR is not set
CONFIG_ARCH_NO_SYSDEV_OPS=y
CONFIG_CONNECTOR=y
CONFIG_PROC_EVENTS=y
# CONFIG_MTD is not set
CONFIG_OF=y

#
# Device Tree and Open Firmware support
#
CONFIG_PROC_DEVICETREE=y
CONFIG_OF_PROMTREE=y
CONFIG_OF_ADDRESS=y
CONFIG_OF_IRQ=y
CONFIG_OF_DEVICE=y
CONFIG_OF_GPIO=y
CONFIG_OF_I2C=y
CONFIG_OF_NET=y
CONFIG_OF_SPI=y
CONFIG_OF_MDIO=y
CONFIG_OF_PCI=y
CONFIG_PARPORT=y
CONFIG_PARPORT_PC=y
CONFIG_PARPORT_SERIAL=y
CONFIG_PARPORT_PC_FIFO=y
CONFIG_PARPORT_PC_SUPERIO=y
CONFIG_PARPORT_PC_PCMCIA=y
# CONFIG_PARPORT_GSC is not set
CONFIG_PARPORT_AX88796=y
CONFIG_PARPORT_1284=y
CONFIG_PARPORT_NOT_PC=y
CONFIG_PNP=y
CONFIG_PNP_DEBUG_MESSAGES=y

#
# Protocols
#
CONFIG_ISAPNP=y
CONFIG_PNPBIOS=y
CONFIG_PNPBIOS_PROC_FS=y
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_FD=y
CONFIG_BLK_DEV_XD=y
# CONFIG_PARIDE is not set
CONFIG_BLK_CPQ_DA=y
CONFIG_BLK_CPQ_CISS_DA=y
CONFIG_CISS_SCSI_TAPE=y
CONFIG_BLK_DEV_DAC960=y
CONFIG_BLK_DEV_UMEM=y
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_CRYPTOLOOP=y
CONFIG_BLK_DEV_DRBD=y
# CONFIG_DRBD_FAULT_INJECTION is not set
CONFIG_BLK_DEV_NBD=y
CONFIG_BLK_DEV_OSD=y
CONFIG_BLK_DEV_SX8=y
CONFIG_BLK_DEV_UB=y
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=16384
CONFIG_BLK_DEV_XIP=y
CONFIG_CDROM_PKTCDVD=y
CONFIG_CDROM_PKTCDVD_BUFFERS=8
CONFIG_CDROM_PKTCDVD_WCACHE=y
CONFIG_ATA_OVER_ETH=y
CONFIG_VIRTIO_BLK=y
CONFIG_BLK_DEV_HD=y
CONFIG_BLK_DEV_RBD=y
CONFIG_SENSORS_LIS3LV02D=y
CONFIG_MISC_DEVICES=y
CONFIG_AD525X_DPOT=m
CONFIG_AD525X_DPOT_I2C=m
CONFIG_AD525X_DPOT_SPI=m
CONFIG_IBM_ASM=y
CONFIG_PHANTOM=y
CONFIG_SGI_IOC4=y
CONFIG_TIFM_CORE=y
CONFIG_TIFM_7XX1=y
CONFIG_ICS932S401=y
CONFIG_ENCLOSURE_SERVICES=y
CONFIG_CS5535_MFGPT=m
CONFIG_CS5535_MFGPT_DEFAULT_IRQ=7
CONFIG_CS5535_CLOCK_EVENT_SRC=m
CONFIG_HP_ILO=y
CONFIG_APDS9802ALS=m
CONFIG_ISL29003=y
CONFIG_ISL29020=y
CONFIG_SENSORS_TSL2550=m
CONFIG_SENSORS_BH1780=m
CONFIG_SENSORS_BH1770=m
CONFIG_SENSORS_APDS990X=m
CONFIG_HMC6352=m
CONFIG_DS1682=m
CONFIG_TI_DAC7512=m
CONFIG_VMWARE_BALLOON=m
CONFIG_BMP085=m
CONFIG_PCH_PHUB=m
CONFIG_C2PORT=y
CONFIG_C2PORT_DURAMAR_2150=y

#
# EEPROM support
#
CONFIG_EEPROM_AT24=y
CONFIG_EEPROM_AT25=y
CONFIG_EEPROM_LEGACY=y
CONFIG_EEPROM_MAX6875=y
CONFIG_EEPROM_93CX6=y
CONFIG_CB710_CORE=y
# CONFIG_CB710_DEBUG is not set
CONFIG_CB710_DEBUG_ASSUMPTIONS=y
CONFIG_IWMC3200TOP=m
# CONFIG_IWMC3200TOP_DEBUG is not set
# CONFIG_IWMC3200TOP_DEBUGFS is not set

#
# Texas Instruments shared transport line discipline #
CONFIG_TI_ST=m
CONFIG_SENSORS_LIS3_I2C=m
CONFIG_HAVE_IDE=y
# CONFIG_IDE is not set

#
# SCSI device support
#
CONFIG_SCSI_MOD=y
CONFIG_RAID_ATTRS=y
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
CONFIG_SCSI_TGT=y
CONFIG_SCSI_NETLINK=y
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=y
CONFIG_CHR_DEV_OSST=y
CONFIG_BLK_DEV_SR=y
CONFIG_BLK_DEV_SR_VENDOR=y
CONFIG_CHR_DEV_SG=y
CONFIG_CHR_DEV_SCH=y
CONFIG_SCSI_ENCLOSURE=y
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
CONFIG_SCSI_SCAN_ASYNC=y
CONFIG_SCSI_WAIT_SCAN=m

#
# SCSI Transports
#
CONFIG_SCSI_SPI_ATTRS=y
CONFIG_SCSI_FC_ATTRS=y
CONFIG_SCSI_FC_TGT_ATTRS=y
CONFIG_SCSI_ISCSI_ATTRS=y
CONFIG_SCSI_SAS_ATTRS=y
CONFIG_SCSI_SAS_LIBSAS=y
CONFIG_SCSI_SAS_ATA=y
CONFIG_SCSI_SAS_HOST_SMP=y
CONFIG_SCSI_SRP_ATTRS=m
CONFIG_SCSI_SRP_TGT_ATTRS=y
CONFIG_SCSI_LOWLEVEL=y
CONFIG_ISCSI_TCP=y
CONFIG_ISCSI_BOOT_SYSFS=y
CONFIG_SCSI_CXGB3_ISCSI=y
CONFIG_SCSI_CXGB4_ISCSI=y
CONFIG_SCSI_BNX2_ISCSI=y
CONFIG_SCSI_BNX2X_FCOE=y
CONFIG_BE2ISCSI=y
CONFIG_BLK_DEV_3W_XXXX_RAID=y
CONFIG_SCSI_HPSA=y
CONFIG_SCSI_3W_9XXX=y
CONFIG_SCSI_3W_SAS=y
CONFIG_SCSI_7000FASST=y
CONFIG_SCSI_ACARD=y
CONFIG_SCSI_AHA152X=y
CONFIG_SCSI_AHA1542=y
CONFIG_SCSI_AHA1740=y
CONFIG_SCSI_AACRAID=y
CONFIG_SCSI_AIC7XXX=y
CONFIG_AIC7XXX_CMDS_PER_DEVICE=4
CONFIG_AIC7XXX_RESET_DELAY_MS=15000
# CONFIG_AIC7XXX_DEBUG_ENABLE is not set
CONFIG_AIC7XXX_DEBUG_MASK=0
# CONFIG_AIC7XXX_REG_PRETTY_PRINT is not set
CONFIG_SCSI_AIC7XXX_OLD=y
CONFIG_SCSI_AIC79XX=y
CONFIG_AIC79XX_CMDS_PER_DEVICE=4
CONFIG_AIC79XX_RESET_DELAY_MS=15000
# CONFIG_AIC79XX_DEBUG_ENABLE is not set
CONFIG_AIC79XX_DEBUG_MASK=0
# CONFIG_AIC79XX_REG_PRETTY_PRINT is not set
CONFIG_SCSI_AIC94XX=y
# CONFIG_AIC94XX_DEBUG is not set
CONFIG_SCSI_MVSAS=y
CONFIG_SCSI_MVSAS_DEBUG=y
CONFIG_SCSI_DPT_I2O=y
CONFIG_SCSI_ADVANSYS=y
CONFIG_SCSI_IN2000=y
CONFIG_SCSI_ARCMSR=y
CONFIG_SCSI_ARCMSR_AER=y
CONFIG_MEGARAID_NEWGEN=y
CONFIG_MEGARAID_MM=y
CONFIG_MEGARAID_MAILBOX=y
CONFIG_MEGARAID_LEGACY=y
CONFIG_MEGARAID_SAS=y
CONFIG_SCSI_MPT2SAS=y
CONFIG_SCSI_MPT2SAS_MAX_SGE=128
CONFIG_SCSI_MPT2SAS_LOGGING=y
CONFIG_SCSI_HPTIOP=y
CONFIG_SCSI_BUSLOGIC=y
CONFIG_SCSI_FLASHPOINT=y
CONFIG_VMWARE_PVSCSI=y
CONFIG_LIBFC=y
CONFIG_LIBFCOE=y
CONFIG_FCOE=y
CONFIG_FCOE_FNIC=y
CONFIG_SCSI_DMX3191D=y
CONFIG_SCSI_DTC3280=y
CONFIG_SCSI_EATA=y
CONFIG_SCSI_EATA_TAGGED_QUEUE=y
CONFIG_SCSI_EATA_LINKED_COMMANDS=y
CONFIG_SCSI_EATA_MAX_TAGS=16
CONFIG_SCSI_FUTURE_DOMAIN=y
CONFIG_SCSI_FD_MCS=y
CONFIG_SCSI_GDTH=y
CONFIG_SCSI_GENERIC_NCR5380=y
CONFIG_SCSI_GENERIC_NCR5380_MMIO=y
CONFIG_SCSI_GENERIC_NCR53C400=y
CONFIG_SCSI_IBMMCA=y
CONFIG_IBMMCA_SCSI_ORDER_STANDARD=y
CONFIG_IBMMCA_SCSI_DEV_RESET=y
CONFIG_SCSI_IPS=y
CONFIG_SCSI_INITIO=y
CONFIG_SCSI_INIA100=y
CONFIG_SCSI_PPA=y
CONFIG_SCSI_IMM=y
CONFIG_SCSI_IZIP_EPP16=y
# CONFIG_SCSI_IZIP_SLOW_CTR is not set
CONFIG_SCSI_NCR53C406A=y
CONFIG_SCSI_NCR_D700=y
CONFIG_SCSI_STEX=y
CONFIG_SCSI_SYM53C8XX_2=y
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16
CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
CONFIG_SCSI_SYM53C8XX_MMIO=y
CONFIG_SCSI_IPR=y
CONFIG_SCSI_IPR_TRACE=y
CONFIG_SCSI_IPR_DUMP=y
CONFIG_SCSI_NCR_Q720=y
CONFIG_SCSI_NCR53C8XX_DEFAULT_TAGS=8
CONFIG_SCSI_NCR53C8XX_MAX_TAGS=32
CONFIG_SCSI_NCR53C8XX_SYNC=20
CONFIG_SCSI_PAS16=y
CONFIG_SCSI_QLOGIC_FAS=y
CONFIG_SCSI_QLOGIC_1280=y
CONFIG_SCSI_QLA_FC=y
CONFIG_SCSI_QLA_ISCSI=y
CONFIG_SCSI_LPFC=y
CONFIG_SCSI_LPFC_DEBUG_FS=y
CONFIG_SCSI_SIM710=y
CONFIG_SCSI_SYM53C416=y
CONFIG_SCSI_DC395x=y
CONFIG_SCSI_DC390T=y
CONFIG_SCSI_T128=y
CONFIG_SCSI_U14_34F=y
CONFIG_SCSI_U14_34F_TAGGED_QUEUE=y
CONFIG_SCSI_U14_34F_LINKED_COMMANDS=y
CONFIG_SCSI_U14_34F_MAX_TAGS=8
CONFIG_SCSI_ULTRASTOR=y
CONFIG_SCSI_NSP32=y
# CONFIG_SCSI_DEBUG is not set
CONFIG_SCSI_PMCRAID=y
CONFIG_SCSI_PM8001=y
CONFIG_SCSI_SRP=y
CONFIG_SCSI_BFA_FC=y
CONFIG_SCSI_LOWLEVEL_PCMCIA=y
CONFIG_PCMCIA_AHA152X=m
CONFIG_PCMCIA_FDOMAIN=m
CONFIG_PCMCIA_NINJA_SCSI=m
CONFIG_PCMCIA_QLOGIC=m
CONFIG_PCMCIA_SYM53C500=m
CONFIG_SCSI_DH=y
CONFIG_SCSI_DH_RDAC=y
CONFIG_SCSI_DH_HP_SW=y
CONFIG_SCSI_DH_EMC=y
CONFIG_SCSI_DH_ALUA=y
CONFIG_SCSI_OSD_INITIATOR=y
CONFIG_SCSI_OSD_ULD=y
CONFIG_SCSI_OSD_DPRINT_SENSE=1
# CONFIG_SCSI_OSD_DEBUG is not set
CONFIG_ATA=y
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_VERBOSE_ERROR=y
CONFIG_ATA_ACPI=y
CONFIG_SATA_PMP=y

#
# Controllers with non-SFF native interface
#
CONFIG_SATA_AHCI=y
CONFIG_SATA_AHCI_PLATFORM=y
CONFIG_SATA_INIC162X=y
CONFIG_SATA_ACARD_AHCI=y
CONFIG_SATA_SIL24=y
CONFIG_ATA_SFF=y

#
# SFF controllers with custom DMA interface
#
CONFIG_PDC_ADMA=y
CONFIG_SATA_QSTOR=y
CONFIG_SATA_SX4=y
CONFIG_ATA_BMDMA=y

#
# SATA SFF controllers with BMDMA
#
CONFIG_ATA_PIIX=y
CONFIG_SATA_MV=y
CONFIG_SATA_NV=y
CONFIG_SATA_PROMISE=y
CONFIG_SATA_SIL=y
CONFIG_SATA_SIS=y
CONFIG_SATA_SVW=y
CONFIG_SATA_ULI=y
CONFIG_SATA_VIA=y
CONFIG_SATA_VITESSE=y

#
# PATA SFF controllers with BMDMA
#
CONFIG_PATA_ALI=y
CONFIG_PATA_AMD=y
CONFIG_PATA_ARASAN_CF=y
CONFIG_PATA_ARTOP=y
CONFIG_PATA_ATIIXP=y
CONFIG_PATA_ATP867X=y
CONFIG_PATA_CMD64X=y
CONFIG_PATA_CS5520=y
CONFIG_PATA_CS5530=y
CONFIG_PATA_CS5535=y
CONFIG_PATA_CS5536=y
CONFIG_PATA_CYPRESS=y
CONFIG_PATA_EFAR=y
CONFIG_PATA_HPT366=y
CONFIG_PATA_HPT37X=y
CONFIG_PATA_HPT3X2N=y
CONFIG_PATA_HPT3X3=y
CONFIG_PATA_HPT3X3_DMA=y
CONFIG_PATA_IT8213=y
CONFIG_PATA_IT821X=y
CONFIG_PATA_JMICRON=y
CONFIG_PATA_MARVELL=y
CONFIG_PATA_NETCELL=y
CONFIG_PATA_NINJA32=y
CONFIG_PATA_NS87415=y
CONFIG_PATA_OLDPIIX=y
CONFIG_PATA_OPTIDMA=y
CONFIG_PATA_PDC2027X=y
CONFIG_PATA_PDC_OLD=y
CONFIG_PATA_RADISYS=y
CONFIG_PATA_RDC=y
CONFIG_PATA_SC1200=y
CONFIG_PATA_SCH=y
CONFIG_PATA_SERVERWORKS=y
CONFIG_PATA_SIL680=y
CONFIG_PATA_SIS=y
CONFIG_PATA_TOSHIBA=y
CONFIG_PATA_TRIFLEX=y
CONFIG_PATA_VIA=y
CONFIG_PATA_WINBOND=y

#
# PIO-only SFF controllers
#
CONFIG_PATA_CMD640_PCI=y
CONFIG_PATA_ISAPNP=y
CONFIG_PATA_MPIIX=y
CONFIG_PATA_NS87410=y
CONFIG_PATA_OPTI=y
CONFIG_PATA_PCMCIA=y
CONFIG_PATA_PLATFORM=y
CONFIG_PATA_QDI=y
CONFIG_PATA_RZ1000=y
CONFIG_PATA_WINBOND_VLB=y

#
# Generic fallback / legacy drivers
#
CONFIG_PATA_ACPI=y
CONFIG_ATA_GENERIC=y
CONFIG_PATA_LEGACY=y
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_AUTODETECT=y
CONFIG_MD_LINEAR=y
CONFIG_MD_RAID0=y
CONFIG_MD_RAID1=y
CONFIG_MD_RAID10=y
CONFIG_MD_RAID456=y
CONFIG_MULTICORE_RAID456=y
CONFIG_MD_MULTIPATH=y
CONFIG_MD_FAULTY=y
CONFIG_BLK_DEV_DM=y
# CONFIG_DM_DEBUG is not set
CONFIG_DM_CRYPT=y
CONFIG_DM_SNAPSHOT=y
CONFIG_DM_MIRROR=y
CONFIG_DM_RAID=m
# CONFIG_DM_LOG_USERSPACE is not set
CONFIG_DM_ZERO=y
CONFIG_DM_MULTIPATH=y
# CONFIG_DM_MULTIPATH_QL is not set
# CONFIG_DM_MULTIPATH_ST is not set
# CONFIG_DM_DELAY is not set
CONFIG_DM_UEVENT=y
CONFIG_DM_FLAKEY=y
CONFIG_TARGET_CORE=m
CONFIG_TCM_IBLOCK=m
CONFIG_TCM_FILEIO=m
CONFIG_TCM_PSCSI=m
CONFIG_LOOPBACK_TARGET=m
# CONFIG_LOOPBACK_TARGET_CDB_DEBUG is not set
CONFIG_FUSION=y
CONFIG_FUSION_SPI=y
CONFIG_FUSION_FC=y
CONFIG_FUSION_SAS=y
CONFIG_FUSION_MAX_SGE=128
CONFIG_FUSION_CTL=y
CONFIG_FUSION_LAN=y
CONFIG_FUSION_LOGGING=y

#
# IEEE 1394 (FireWire) support
#
CONFIG_FIREWIRE=y
CONFIG_FIREWIRE_OHCI=y
CONFIG_FIREWIRE_OHCI_DEBUG=y
CONFIG_FIREWIRE_SBP2=y
CONFIG_FIREWIRE_NET=y
CONFIG_FIREWIRE_NOSY=y
CONFIG_I2O=y
CONFIG_I2O_LCT_NOTIFY_ON_CHANGES=y
CONFIG_I2O_EXT_ADAPTEC=y
CONFIG_I2O_CONFIG=y
CONFIG_I2O_CONFIG_OLD_IOCTL=y
CONFIG_I2O_BUS=y
CONFIG_I2O_BLOCK=y
CONFIG_I2O_SCSI=y
CONFIG_I2O_PROC=y
CONFIG_MACINTOSH_DRIVERS=y
CONFIG_MAC_EMUMOUSEBTN=y
CONFIG_NETDEVICES=y
CONFIG_IFB=m
CONFIG_DUMMY=m
CONFIG_BONDING=m
CONFIG_MACVLAN=m
CONFIG_MACVTAP=m
CONFIG_EQUALIZER=m
CONFIG_TUN=m
CONFIG_VETH=m
CONFIG_NET_SB1000=m
CONFIG_ARCNET=m
CONFIG_ARCNET_1201=m
CONFIG_ARCNET_1051=m
CONFIG_ARCNET_RAW=m
CONFIG_ARCNET_CAP=m
CONFIG_ARCNET_COM90xx=m
CONFIG_ARCNET_COM90xxIO=m
CONFIG_ARCNET_RIM_I=m
CONFIG_ARCNET_COM20020=m
CONFIG_ARCNET_COM20020_ISA=m
CONFIG_ARCNET_COM20020_PCI=m
CONFIG_MII=y
CONFIG_PHYLIB=y

#
# MII PHY device drivers
#
CONFIG_MARVELL_PHY=m
CONFIG_DAVICOM_PHY=m
CONFIG_QSEMI_PHY=m
CONFIG_LXT_PHY=m
CONFIG_CICADA_PHY=m
CONFIG_VITESSE_PHY=m
CONFIG_SMSC_PHY=m
CONFIG_BROADCOM_PHY=m
CONFIG_BCM63XX_PHY=m
CONFIG_ICPLUS_PHY=m
CONFIG_REALTEK_PHY=m
CONFIG_NATIONAL_PHY=m
CONFIG_STE10XP=m
CONFIG_LSI_ET1011C_PHY=m CONFIG_MICREL_PHY=m
CONFIG_FIXED_PHY=y
CONFIG_MDIO_BITBANG=m
CONFIG_MDIO_GPIO=m
CONFIG_NET_ETHERNET=y
CONFIG_HAPPYMEAL=m
CONFIG_SUNGEM=m
CONFIG_CASSINI=m
CONFIG_NET_VENDOR_3COM=y
CONFIG_EL1=m
CONFIG_EL2=m
CONFIG_ELPLUS=m
CONFIG_EL16=m
CONFIG_EL3=m
CONFIG_3C515=m
CONFIG_ELMC=m
CONFIG_ELMC_II=m
CONFIG_VORTEX=y
CONFIG_TYPHOON=m
CONFIG_LANCE=m
CONFIG_NET_VENDOR_SMC=y
CONFIG_WD80x3=m
CONFIG_ULTRAMCA=m
CONFIG_ULTRA=m
CONFIG_ULTRA32=m
CONFIG_SMC9194=m
CONFIG_ENC28J60=m
# CONFIG_ENC28J60_WRITEVERIFY is not set CONFIG_ETHOC=m
CONFIG_NET_VENDOR_RACAL=y
CONFIG_NI52=m
CONFIG_NI65=m
CONFIG_DNET=m
CONFIG_NET_TULIP=y
CONFIG_DE2104X=m
CONFIG_DE2104X_DSL=0
CONFIG_TULIP=m
CONFIG_TULIP_MWI=y
CONFIG_TULIP_MMIO=y
CONFIG_TULIP_NAPI=y CONFIG_TULIP_NAPI_HW_MITIGATION=y CONFIG_DE4X5=m
CONFIG_WINBOND_840=m
CONFIG_DM9102=m
CONFIG_ULI526X=m
CONFIG_PCMCIA_XIRCOM=m
CONFIG_AT1700=m
CONFIG_DEPCA=m
CONFIG_HP100=m
CONFIG_NET_ISA=y
CONFIG_E2100=m
CONFIG_EWRK3=m
CONFIG_EEXPRESS=m
CONFIG_EEXPRESS_PRO=m
CONFIG_HPLAN_PLUS=m
CONFIG_HPLAN=m
CONFIG_LP486E=m
CONFIG_ETH16I=m
CONFIG_NE2000=m
CONFIG_ZNET=m
CONFIG_SEEQ8005=m
CONFIG_NE2_MCA=m
CONFIG_IBMLANA=m
# CONFIG_IBM_NEW_EMAC_ZMII is not set
# CONFIG_IBM_NEW_EMAC_RGMII is not set
# CONFIG_IBM_NEW_EMAC_TAH is not set
# CONFIG_IBM_NEW_EMAC_EMAC4 is not set
# CONFIG_IBM_NEW_EMAC_NO_FLOW_CTRL is not set # CONFIG_IBM_NEW_EMAC_MAL_CLR_ICINTSTAT is not set # CONFIG_IBM_NEW_EMAC_MAL_COMMON_ERR is not set CONFIG_NET_PCI=y
CONFIG_PCNET32=m
CONFIG_AMD8111_ETH=m CONFIG_ADAPTEC_STARFIRE=m
CONFIG_AC3200=m
CONFIG_KSZ884X_PCI=m
CONFIG_APRICOT=m
CONFIG_B44=m
CONFIG_B44_PCI_AUTOSELECT=y CONFIG_B44_PCICORE_AUTOSELECT=y CONFIG_B44_PCI=y
CONFIG_FORCEDETH=y
CONFIG_CS89x0=m
CONFIG_E100=y
CONFIG_LNE390=m
CONFIG_FEALNX=m
CONFIG_NATSEMI=m
CONFIG_NE2K_PCI=m
CONFIG_NE3210=m
CONFIG_ES3210=m
CONFIG_8139CP=m
CONFIG_8139TOO=y
CONFIG_8139TOO_PIO=y CONFIG_8139TOO_TUNE_TWISTER=y CONFIG_8139TOO_8129=y
# CONFIG_8139_OLD_RX_RESET is not set CONFIG_R6040=m
CONFIG_SIS900=m
CONFIG_EPIC100=m
CONFIG_SMSC9420=m
CONFIG_SUNDANCE=m
# CONFIG_SUNDANCE_MMIO is not set CONFIG_TLAN=m
CONFIG_KS8842=m
CONFIG_KS8851=m
CONFIG_KS8851_MLL=m
CONFIG_VIA_RHINE=y
CONFIG_VIA_RHINE_MMIO=y
CONFIG_SC92031=m
CONFIG_NET_POCKET=y
CONFIG_ATP=y
CONFIG_DE600=y
CONFIG_DE620=y
CONFIG_ATL2=y
CONFIG_NETDEV_1000=y
CONFIG_ACENIC=m
# CONFIG_ACENIC_OMIT_TIGON_I is not set CONFIG_DL2K=m
CONFIG_E1000=m
CONFIG_E1000E=y
CONFIG_IP1000=m
CONFIG_IGB=m
CONFIG_IGB_DCA=y
CONFIG_IGBVF=m
CONFIG_NS83820=m
CONFIG_HAMACHI=m
CONFIG_YELLOWFIN=m
CONFIG_R8169=y
CONFIG_SIS190=m
CONFIG_SKGE=y
# CONFIG_SKGE_DEBUG is not set CONFIG_SKY2=m
# CONFIG_SKY2_DEBUG is not set CONFIG_VIA_VELOCITY=m
CONFIG_TIGON3=y
CONFIG_BNX2=y
CONFIG_CNIC=y
CONFIG_QLA3XXX=m
CONFIG_ATL1=m
CONFIG_ATL1E=m
CONFIG_ATL1C=m
CONFIG_JME=m
CONFIG_STMMAC_ETH=m
CONFIG_STMMAC_DA=y CONFIG_STMMAC_DUAL_MAC=y
CONFIG_PCH_GBE=m
CONFIG_NETDEV_10000=y
CONFIG_MDIO=y
CONFIG_CHELSIO_T1=m
CONFIG_CHELSIO_T1_1G=y
CONFIG_CHELSIO_T3=y
CONFIG_CHELSIO_T4=y
CONFIG_CHELSIO_T4VF=m
CONFIG_ENIC=m
CONFIG_IXGBE=m
CONFIG_IXGBE_DCA=y
CONFIG_IXGBE_DCB=y
CONFIG_IXGBEVF=m
CONFIG_IXGB=m
CONFIG_S2IO=m
CONFIG_VXGE=m
# CONFIG_VXGE_DEBUG_TRACE_ALL is not set CONFIG_MYRI10GE=m
CONFIG_MYRI10GE_DCA=y
CONFIG_NETXEN_NIC=m
CONFIG_NIU=m
CONFIG_MLX4_EN=m
CONFIG_MLX4_CORE=m
CONFIG_MLX4_DEBUG=y
CONFIG_TEHUTI=m
CONFIG_BNX2X=m
CONFIG_QLCNIC=m
CONFIG_QLGE=m
CONFIG_BNA=m
CONFIG_SFC=m
CONFIG_BE2NET=m
CONFIG_TR=y
CONFIG_IBMTR=m
CONFIG_IBMOL=m
CONFIG_IBMLS=m
CONFIG_3C359=m
CONFIG_TMS380TR=m
CONFIG_TMSPCI=m
CONFIG_SKISA=m
CONFIG_PROTEON=m
CONFIG_ABYSS=m
CONFIG_MADGEMC=m
CONFIG_SMCTR=m
CONFIG_WLAN=y
CONFIG_PCMCIA_RAYCS=m CONFIG_LIBERTAS_THINFIRM=m
# CONFIG_LIBERTAS_THINFIRM_DEBUG is not set CONFIG_LIBERTAS_THINFIRM_USB=m CONFIG_AIRO=m
CONFIG_ATMEL=m
CONFIG_PCI_ATMEL=m
CONFIG_PCMCIA_ATMEL=m CONFIG_AT76C50X_USB=m
CONFIG_AIRO_CS=m
CONFIG_PCMCIA_WL3501=m
CONFIG_PRISM54=m
CONFIG_USB_ZD1201=m CONFIG_USB_NET_RNDIS_WLAN=m CONFIG_RTL8180=m
CONFIG_RTL8187=m
CONFIG_RTL8187_LEDS=y
CONFIG_ADM8211=m
CONFIG_MAC80211_HWSIM=m
CONFIG_MWL8K=m
CONFIG_ATH_COMMON=m
# CONFIG_ATH_DEBUG is not set CONFIG_ATH5K=m
# CONFIG_ATH5K_DEBUG is not set
# CONFIG_ATH5K_TRACER is not set CONFIG_ATH5K_PCI=y
CONFIG_ATH9K_HW=m
CONFIG_ATH9K_COMMON=m
CONFIG_ATH9K=m
# CONFIG_ATH9K_DEBUGFS is not set CONFIG_ATH9K_RATE_CONTROL=y CONFIG_ATH9K_HTC=m
# CONFIG_ATH9K_HTC_DEBUGFS is not set CONFIG_AR9170_USB=m
CONFIG_AR9170_LEDS=y
# CONFIG_CARL9170 is not set
CONFIG_B43=m
CONFIG_B43_PCI_AUTOSELECT=y CONFIG_B43_PCICORE_AUTOSELECT=y CONFIG_B43_PCMCIA=y
CONFIG_B43_SDIO=y
CONFIG_B43_PIO=y
CONFIG_B43_PHY_N=y
CONFIG_B43_PHY_LP=y
CONFIG_B43_LEDS=y
CONFIG_B43_HWRNG=y
# CONFIG_B43_DEBUG is not set CONFIG_B43LEGACY=m CONFIG_B43LEGACY_PCI_AUTOSELECT=y CONFIG_B43LEGACY_PCICORE_AUTOSELECT=y CONFIG_B43LEGACY_LEDS=y CONFIG_B43LEGACY_HWRNG=y
# CONFIG_B43LEGACY_DEBUG is not set CONFIG_B43LEGACY_DMA=y CONFIG_B43LEGACY_PIO=y CONFIG_B43LEGACY_DMA_AND_PIO_MODE=y
# CONFIG_B43LEGACY_DMA_MODE is not set
# CONFIG_B43LEGACY_PIO_MODE is not set CONFIG_HOSTAP=y
CONFIG_HOSTAP_FIRMWARE=y CONFIG_HOSTAP_FIRMWARE_NVRAM=y CONFIG_HOSTAP_PLX=m
CONFIG_HOSTAP_PCI=m
CONFIG_HOSTAP_CS=m
CONFIG_IPW2100=m
CONFIG_IPW2100_MONITOR=y
# CONFIG_IPW2100_DEBUG is not set CONFIG_IPW2200=m
CONFIG_IPW2200_MONITOR=y CONFIG_IPW2200_RADIOTAP=y CONFIG_IPW2200_PROMISCUOUS=y CONFIG_IPW2200_QOS=y
# CONFIG_IPW2200_DEBUG is not set CONFIG_LIBIPW=m
# CONFIG_LIBIPW_DEBUG is not set CONFIG_IWLAGN=m

#
# Debugging Options
#
# CONFIG_IWLWIFI_DEBUG is not set
CONFIG_IWLWIFI_DEVICE_TRACING=y
CONFIG_IWL_P2P=y
CONFIG_IWLWIFI_LEGACY=m

#
# Debugging Options
#
# CONFIG_IWLWIFI_LEGACY_DEBUG is not set
# CONFIG_IWLWIFI_LEGACY_DEVICE_TRACING is not set
CONFIG_IWL4965=m
CONFIG_IWL3945=m
CONFIG_IWM=m
# CONFIG_IWM_DEBUG is not set
CONFIG_IWM_TRACING=y
CONFIG_LIBERTAS=m
CONFIG_LIBERTAS_USB=m
CONFIG_LIBERTAS_CS=m
CONFIG_LIBERTAS_SDIO=m
CONFIG_LIBERTAS_SPI=m
# CONFIG_LIBERTAS_DEBUG is not set
CONFIG_LIBERTAS_MESH=y
CONFIG_HERMES=m
CONFIG_HERMES_PRISM=y
CONFIG_HERMES_CACHE_FW_ON_INIT=y
CONFIG_PLX_HERMES=m
CONFIG_TMD_HERMES=m
CONFIG_NORTEL_HERMES=m
CONFIG_PCI_HERMES=m
CONFIG_PCMCIA_HERMES=m
CONFIG_PCMCIA_SPECTRUM=m
CONFIG_ORINOCO_USB=m
CONFIG_P54_COMMON=m
CONFIG_P54_USB=m
CONFIG_P54_PCI=m
CONFIG_P54_SPI=m
CONFIG_P54_SPI_DEFAULT_EEPROM=y
CONFIG_P54_LEDS=y
CONFIG_RT2X00=m
CONFIG_RT2400PCI=m
CONFIG_RT2500PCI=m
CONFIG_RT61PCI=m
CONFIG_RT2800PCI=m
CONFIG_RT2800PCI_RT33XX=y
CONFIG_RT2800PCI_RT35XX=y
CONFIG_RT2800PCI_RT53XX=y
CONFIG_RT2500USB=m
CONFIG_RT73USB=m
CONFIG_RT2800USB=m
CONFIG_RT2800USB_RT33XX=y
CONFIG_RT2800USB_RT35XX=y
CONFIG_RT2800USB_UNKNOWN=y
CONFIG_RT2800_LIB=m
CONFIG_RT2X00_LIB_PCI=m
CONFIG_RT2X00_LIB_USB=m
CONFIG_RT2X00_LIB=m
CONFIG_RT2X00_LIB_HT=y
CONFIG_RT2X00_LIB_FIRMWARE=y
CONFIG_RT2X00_LIB_CRYPTO=y
CONFIG_RT2X00_LIB_LEDS=y
# CONFIG_RT2X00_DEBUG is not set
CONFIG_RTL8192CE=m
CONFIG_RTL8192CU=m
CONFIG_RTLWIFI=m
CONFIG_RTL8192C_COMMON=m
CONFIG_WL1251=m
CONFIG_WL1251_SPI=m
CONFIG_WL1251_SDIO=m
CONFIG_WL12XX_MENU=m
CONFIG_WL12XX=m
CONFIG_WL12XX_HT=y
CONFIG_WL12XX_SPI=m
CONFIG_WL12XX_SDIO=m
CONFIG_WL12XX_SDIO_TEST=m
CONFIG_WL12XX_PLATFORM_DATA=y
CONFIG_ZD1211RW=m
# CONFIG_ZD1211RW_DEBUG is not set

#
# WiMAX Wireless Broadband devices
#
CONFIG_WIMAX_I2400M=m
CONFIG_WIMAX_I2400M_USB=m
CONFIG_WIMAX_I2400M_SDIO=m
CONFIG_WIMAX_IWMC3200_SDIO=y
CONFIG_WIMAX_I2400M_DEBUG_LEVEL=8

#
# USB Network Adapters
#
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
CONFIG_USB_PEGASUS=m
CONFIG_USB_RTL8150=m
CONFIG_USB_USBNET=m
CONFIG_USB_NET_AX8817X=m
CONFIG_USB_NET_CDCETHER=m
CONFIG_USB_NET_CDC_EEM=m
CONFIG_USB_NET_CDC_NCM=m
CONFIG_USB_NET_DM9601=m
CONFIG_USB_NET_SMSC75XX=m
CONFIG_USB_NET_SMSC95XX=m
CONFIG_USB_NET_GL620A=m
CONFIG_USB_NET_NET1080=m
CONFIG_USB_NET_PLUSB=m
CONFIG_USB_NET_MCS7830=m
CONFIG_USB_NET_RNDIS_HOST=m
CONFIG_USB_NET_CDC_SUBSET=m
CONFIG_USB_ALI_M5632=y
CONFIG_USB_AN2720=y
CONFIG_USB_BELKIN=y
CONFIG_USB_ARMLINUX=y
CONFIG_USB_EPSON2888=y
CONFIG_USB_KC2190=y
CONFIG_USB_NET_ZAURUS=m
CONFIG_USB_NET_CX82310_ETH=m
CONFIG_USB_HSO=m
CONFIG_USB_NET_INT51X1=m
CONFIG_USB_CDC_PHONET=m
CONFIG_USB_IPHETH=m
CONFIG_USB_SIERRA_NET=m
CONFIG_USB_VL600=m
CONFIG_NET_PCMCIA=y
CONFIG_PCMCIA_3C589=m
CONFIG_PCMCIA_3C574=m
CONFIG_PCMCIA_FMVJ18X=m
CONFIG_PCMCIA_PCNET=m
CONFIG_PCMCIA_NMCLAN=m
CONFIG_PCMCIA_SMC91C92=m
CONFIG_PCMCIA_XIRC2PS=m
CONFIG_PCMCIA_AXNET=m
CONFIG_ARCNET_COM20020_CS=m
CONFIG_PCMCIA_IBMTR=m
CONFIG_WAN=y
CONFIG_HOSTESS_SV11=m
CONFIG_COSA=m
CONFIG_LANMEDIA=m
CONFIG_SEALEVEL_4021=m
CONFIG_HDLC=m
CONFIG_HDLC_RAW=m
CONFIG_HDLC_RAW_ETH=m
CONFIG_HDLC_CISCO=m
CONFIG_HDLC_FR=m
CONFIG_HDLC_PPP=m
CONFIG_HDLC_X25=m
CONFIG_PCI200SYN=m
CONFIG_WANXL=m
CONFIG_PC300TOO=m
CONFIG_N2=m
CONFIG_C101=m
CONFIG_FARSYNC=m
CONFIG_DSCC4=m
CONFIG_DSCC4_PCISYNC=y
CONFIG_DSCC4_PCI_RST=y
CONFIG_DLCI=m
CONFIG_DLCI_MAX=8
CONFIG_SDLA=m
CONFIG_WAN_ROUTER_DRIVERS=m
CONFIG_CYCLADES_SYNC=m
CONFIG_CYCLOMX_X25=y
CONFIG_LAPBETHER=m
CONFIG_X25_ASY=m
CONFIG_SBNI=m
CONFIG_SBNI_MULTILINE=y
CONFIG_ATM_DRIVERS=y
# CONFIG_ATM_DUMMY is not set
CONFIG_ATM_TCP=m
CONFIG_ATM_LANAI=m
CONFIG_ATM_ENI=m
# CONFIG_ATM_ENI_DEBUG is not set
# CONFIG_ATM_ENI_TUNE_BURST is not set
CONFIG_ATM_FIRESTREAM=m
CONFIG_ATM_ZATM=m
# CONFIG_ATM_ZATM_DEBUG is not set
CONFIG_ATM_NICSTAR=m
# CONFIG_ATM_NICSTAR_USE_SUNI is not set
CONFIG_ATM_NICSTAR_USE_IDT77105=y
CONFIG_ATM_IDT77252=m
# CONFIG_ATM_IDT77252_DEBUG is not set
# CONFIG_ATM_IDT77252_RCV_ALL is not set
CONFIG_ATM_IDT77252_USE_SUNI=y
CONFIG_ATM_AMBASSADOR=m
# CONFIG_ATM_AMBASSADOR_DEBUG is not set
CONFIG_ATM_HORIZON=m
# CONFIG_ATM_HORIZON_DEBUG is not set
CONFIG_ATM_IA=m
# CONFIG_ATM_IA_DEBUG is not set
CONFIG_ATM_FORE200E=m
CONFIG_ATM_FORE200E_USE_TASKLET=y
CONFIG_ATM_FORE200E_TX_RETRY=16
CONFIG_ATM_FORE200E_DEBUG=0
CONFIG_ATM_HE=m
# CONFIG_ATM_HE_USE_SUNI is not set
CONFIG_ATM_SOLOS=m
CONFIG_IEEE802154_DRIVERS=m
CONFIG_IEEE802154_FAKEHARD=m

#
# CAIF transport drivers
#
CONFIG_CAIF_TTY=m
CONFIG_CAIF_SPI_SLAVE=m
CONFIG_CAIF_SPI_SYNC=y
CONFIG_RIONET=m
CONFIG_RIONET_TX_SIZE=128
CONFIG_RIONET_RX_SIZE=128
CONFIG_FDDI=y
CONFIG_DEFXX=m
# CONFIG_DEFXX_MMIO is not set
CONFIG_SKFP=m
CONFIG_HIPPI=y
CONFIG_ROADRUNNER=m
# CONFIG_ROADRUNNER_LARGE_RINGS is not set
CONFIG_PLIP=y
CONFIG_PPP=y
CONFIG_PPP_MULTILINK=y
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=y
CONFIG_PPP_SYNC_TTY=y
CONFIG_PPP_DEFLATE=y
CONFIG_PPP_BSDCOMP=y
CONFIG_PPP_MPPE=y
CONFIG_PPPOE=y
CONFIG_PPTP=m
CONFIG_PPPOATM=y
CONFIG_PPPOL2TP=m
CONFIG_SLIP=y
CONFIG_SLIP_COMPRESSED=y
CONFIG_SLHC=y
CONFIG_SLIP_SMART=y
CONFIG_SLIP_MODE_SLIP6=y
CONFIG_NET_FC=y
CONFIG_NETCONSOLE=y
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_NETPOLL=y
CONFIG_NETPOLL_TRAP=y
CONFIG_NET_POLL_CONTROLLER=y
CONFIG_VIRTIO_NET=m
CONFIG_VMXNET3=m
CONFIG_ISDN=y
CONFIG_ISDN_I4L=m
CONFIG_ISDN_PPP=y
CONFIG_ISDN_PPP_VJ=y
CONFIG_ISDN_MPP=y
CONFIG_IPPP_FILTER=y
CONFIG_ISDN_PPP_BSDCOMP=m
CONFIG_ISDN_AUDIO=y
CONFIG_ISDN_TTY_FAX=y
CONFIG_ISDN_X25=y

#
# ISDN feature submodules
#
CONFIG_ISDN_DIVERSION=m

#
# ISDN4Linux hardware drivers
#

#
# Passive cards
#
CONFIG_ISDN_DRV_HISAX=m

#
# D-channel protocol features
#
CONFIG_HISAX_EURO=y
CONFIG_DE_AOC=y
CONFIG_HISAX_NO_SENDCOMPLETE=y
CONFIG_HISAX_NO_LLC=y
CONFIG_HISAX_NO_KEYPAD=y
CONFIG_HISAX_1TR6=y
CONFIG_HISAX_NI1=y
CONFIG_HISAX_MAX_CARDS=8

#
# HiSax supported cards
#
CONFIG_HISAX_16_0=y
CONFIG_HISAX_16_3=y
CONFIG_HISAX_TELESPCI=y
CONFIG_HISAX_S0BOX=y
CONFIG_HISAX_AVM_A1=y
CONFIG_HISAX_FRITZPCI=y
CONFIG_HISAX_AVM_A1_PCMCIA=y
CONFIG_HISAX_ELSA=y
CONFIG_HISAX_IX1MICROR2=y
CONFIG_HISAX_DIEHLDIVA=y
CONFIG_HISAX_ASUSCOM=y
CONFIG_HISAX_TELEINT=y
CONFIG_HISAX_HFCS=y
CONFIG_HISAX_SEDLBAUER=y
CONFIG_HISAX_SPORTSTER=y
CONFIG_HISAX_MIC=y
CONFIG_HISAX_NETJET=y
CONFIG_HISAX_NETJET_U=y
CONFIG_HISAX_NICCY=y
CONFIG_HISAX_ISURF=y
CONFIG_HISAX_HSTSAPHIR=y
CONFIG_HISAX_BKM_A4T=y
CONFIG_HISAX_SCT_QUADRO=y
CONFIG_HISAX_GAZEL=y
CONFIG_HISAX_HFC_PCI=y
CONFIG_HISAX_W6692=y
CONFIG_HISAX_HFC_SX=y
CONFIG_HISAX_ENTERNOW_PCI=y
# CONFIG_HISAX_DEBUG is not set

#
# HiSax PCMCIA card service modules
#
CONFIG_HISAX_SEDLBAUER_CS=m
CONFIG_HISAX_ELSA_CS=m
CONFIG_HISAX_AVM_A1_CS=m
CONFIG_HISAX_TELES_CS=m

#
# HiSax sub driver modules
#
CONFIG_HISAX_ST5481=m
CONFIG_HISAX_HFCUSB=m
CONFIG_HISAX_HFC4S8S=m
CONFIG_HISAX_FRITZ_PCIPNP=m

#
# Active cards
#
# CONFIG_ISDN_DRV_ICN is not set
CONFIG_ISDN_DRV_PCBIT=m
CONFIG_ISDN_DRV_SC=m
CONFIG_ISDN_DRV_ACT2000=m
CONFIG_ISDN_CAPI=m
CONFIG_ISDN_DRV_AVMB1_VERBOSE_REASON=y
CONFIG_CAPI_TRACE=y
CONFIG_ISDN_CAPI_MIDDLEWARE=y
CONFIG_ISDN_CAPI_CAPI20=m
CONFIG_ISDN_CAPI_CAPIFS_BOOL=y
CONFIG_ISDN_CAPI_CAPIFS=m
CONFIG_ISDN_CAPI_CAPIDRV=m

#
# CAPI hardware drivers
#
CONFIG_CAPI_AVM=y
CONFIG_ISDN_DRV_AVMB1_B1ISA=m
CONFIG_ISDN_DRV_AVMB1_B1PCI=m
CONFIG_ISDN_DRV_AVMB1_B1PCIV4=y
CONFIG_ISDN_DRV_AVMB1_T1ISA=m
CONFIG_ISDN_DRV_AVMB1_B1PCMCIA=m
CONFIG_ISDN_DRV_AVMB1_AVM_CS=m
CONFIG_ISDN_DRV_AVMB1_T1PCI=m
CONFIG_ISDN_DRV_AVMB1_C4=m
CONFIG_CAPI_EICON=y
CONFIG_ISDN_DIVAS=m
CONFIG_ISDN_DIVAS_BRIPCI=y
CONFIG_ISDN_DIVAS_PRIPCI=y
CONFIG_ISDN_DIVAS_DIVACAPI=m
CONFIG_ISDN_DIVAS_USERIDI=m
CONFIG_ISDN_DIVAS_MAINT=m
CONFIG_ISDN_DRV_GIGASET=m
CONFIG_GIGASET_CAPI=y
# CONFIG_GIGASET_I4L is not set
# CONFIG_GIGASET_DUMMYLL is not set
CONFIG_GIGASET_BASE=m
CONFIG_GIGASET_M105=m
CONFIG_GIGASET_M101=m
# CONFIG_GIGASET_DEBUG is not set
CONFIG_HYSDN=m
CONFIG_HYSDN_CAPI=y
CONFIG_MISDN=m
CONFIG_MISDN_DSP=m
CONFIG_MISDN_L1OIP=m

#
# mISDN hardware drivers
#
CONFIG_MISDN_HFCPCI=m
CONFIG_MISDN_HFCMULTI=m
CONFIG_MISDN_HFCUSB=m
CONFIG_MISDN_AVMFRITZ=m
CONFIG_MISDN_SPEEDFAX=m
CONFIG_MISDN_INFINEON=m
CONFIG_MISDN_W6692=m
CONFIG_MISDN_NETJET=m
CONFIG_MISDN_IPAC=m
CONFIG_MISDN_ISAR=m
CONFIG_ISDN_HDLC=m
CONFIG_PHONE=m
CONFIG_PHONE_IXJ=m
CONFIG_PHONE_IXJ_PCMCIA=m

#
# Input device support
#
CONFIG_INPUT=y
CONFIG_INPUT_FF_MEMLESS=y
CONFIG_INPUT_POLLDEV=y
CONFIG_INPUT_SPARSEKMAP=y

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=y
CONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ADP5520=y
CONFIG_KEYBOARD_ADP5588=y
CONFIG_KEYBOARD_ATKBD=y
CONFIG_KEYBOARD_QT1070=y
CONFIG_KEYBOARD_QT2160=y
CONFIG_KEYBOARD_LKKBD=y
CONFIG_KEYBOARD_GPIO=y
CONFIG_KEYBOARD_GPIO_POLLED=y
CONFIG_KEYBOARD_TCA6416=y
CONFIG_KEYBOARD_MATRIX=y
CONFIG_KEYBOARD_LM8323=y
CONFIG_KEYBOARD_MAX7359=y
CONFIG_KEYBOARD_MCS=y
CONFIG_KEYBOARD_NEWTON=y
CONFIG_KEYBOARD_OPENCORES=y
CONFIG_KEYBOARD_STOWAWAY=y
CONFIG_KEYBOARD_SUNKBD=y
CONFIG_KEYBOARD_STMPE=y
CONFIG_KEYBOARD_TC3589X=y
CONFIG_KEYBOARD_TWL4030=y
CONFIG_KEYBOARD_XTKBD=y
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
CONFIG_MOUSE_PS2_ALPS=y
CONFIG_MOUSE_PS2_LOGIPS2PP=y
CONFIG_MOUSE_PS2_SYNAPTICS=y
CONFIG_MOUSE_PS2_LIFEBOOK=y
CONFIG_MOUSE_PS2_TRACKPOINT=y
CONFIG_MOUSE_PS2_ELANTECH=y
CONFIG_MOUSE_PS2_SENTELIC=y
CONFIG_MOUSE_PS2_TOUCHKIT=y
CONFIG_MOUSE_PS2_OLPC=y
CONFIG_MOUSE_SERIAL=y
CONFIG_MOUSE_APPLETOUCH=y
CONFIG_MOUSE_BCM5974=y
CONFIG_MOUSE_INPORT=y
CONFIG_MOUSE_ATIXL=y
CONFIG_MOUSE_LOGIBM=y
CONFIG_MOUSE_PC110PAD=y
CONFIG_MOUSE_VSXXXAA=y
CONFIG_MOUSE_GPIO=y
CONFIG_MOUSE_SYNAPTICS_I2C=y
CONFIG_INPUT_JOYSTICK=y
CONFIG_JOYSTICK_ANALOG=y
CONFIG_JOYSTICK_A3D=m
CONFIG_JOYSTICK_ADI=m
CONFIG_JOYSTICK_COBRA=m
CONFIG_JOYSTICK_GF2K=m
CONFIG_JOYSTICK_GRIP=m
CONFIG_JOYSTICK_GRIP_MP=m
CONFIG_JOYSTICK_GUILLEMOT=m
CONFIG_JOYSTICK_INTERACT=m
CONFIG_JOYSTICK_SIDEWINDER=m
CONFIG_JOYSTICK_TMDC=m
CONFIG_JOYSTICK_IFORCE=m
CONFIG_JOYSTICK_IFORCE_USB=y
CONFIG_JOYSTICK_IFORCE_232=y
CONFIG_JOYSTICK_WARRIOR=m
CONFIG_JOYSTICK_MAGELLAN=m
CONFIG_JOYSTICK_SPACEORB=m
CONFIG_JOYSTICK_SPACEBALL=m
CONFIG_JOYSTICK_STINGER=m
CONFIG_JOYSTICK_TWIDJOY=m
CONFIG_JOYSTICK_ZHENHUA=m
CONFIG_JOYSTICK_DB9=m
CONFIG_JOYSTICK_GAMECON=m
CONFIG_JOYSTICK_TURBOGRAFX=m
CONFIG_JOYSTICK_AS5011=m
CONFIG_JOYSTICK_JOYDUMP=m
CONFIG_JOYSTICK_XPAD=m
CONFIG_JOYSTICK_XPAD_FF=y
CONFIG_JOYSTICK_XPAD_LEDS=y
CONFIG_JOYSTICK_WALKERA0701=m
CONFIG_INPUT_TABLET=y
CONFIG_TABLET_USB_ACECAD=y
CONFIG_TABLET_USB_AIPTEK=y
CONFIG_TABLET_USB_GTCO=y
CONFIG_TABLET_USB_HANWANG=y
CONFIG_TABLET_USB_KBTAB=y
CONFIG_TABLET_USB_WACOM=y
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_88PM860X=y
CONFIG_TOUCHSCREEN_ADS7846=y
CONFIG_TOUCHSCREEN_AD7877=y
CONFIG_TOUCHSCREEN_AD7879=y
CONFIG_TOUCHSCREEN_AD7879_I2C=y
CONFIG_TOUCHSCREEN_AD7879_SPI=y
CONFIG_TOUCHSCREEN_ATMEL_MXT=y
CONFIG_TOUCHSCREEN_BU21013=y
CONFIG_TOUCHSCREEN_CY8CTMG110=y
CONFIG_TOUCHSCREEN_DA9034=y
CONFIG_TOUCHSCREEN_DYNAPRO=y
CONFIG_TOUCHSCREEN_HAMPSHIRE=y
CONFIG_TOUCHSCREEN_EETI=y
CONFIG_TOUCHSCREEN_FUJITSU=y
CONFIG_TOUCHSCREEN_GUNZE=y
CONFIG_TOUCHSCREEN_ELO=y
CONFIG_TOUCHSCREEN_WACOM_W8001=y
CONFIG_TOUCHSCREEN_MCS5000=y
CONFIG_TOUCHSCREEN_MTOUCH=y
CONFIG_TOUCHSCREEN_INEXIO=y
CONFIG_TOUCHSCREEN_MK712=y
CONFIG_TOUCHSCREEN_HTCPEN=y
CONFIG_TOUCHSCREEN_PENMOUNT=y
CONFIG_TOUCHSCREEN_TOUCHRIGHT=y
CONFIG_TOUCHSCREEN_TOUCHWIN=y
CONFIG_TOUCHSCREEN_UCB1400=m
CONFIG_TOUCHSCREEN_WM831X=y
CONFIG_TOUCHSCREEN_WM97XX=m
CONFIG_TOUCHSCREEN_WM9705=y
CONFIG_TOUCHSCREEN_WM9712=y
CONFIG_TOUCHSCREEN_WM9713=y
CONFIG_TOUCHSCREEN_USB_COMPOSITE=y
CONFIG_TOUCHSCREEN_MC13783=m
CONFIG_TOUCHSCREEN_USB_EGALAX=y
CONFIG_TOUCHSCREEN_USB_PANJIT=y
CONFIG_TOUCHSCREEN_USB_3M=y
CONFIG_TOUCHSCREEN_USB_ITM=y
CONFIG_TOUCHSCREEN_USB_ETURBO=y
CONFIG_TOUCHSCREEN_USB_GUNZE=y
CONFIG_TOUCHSCREEN_USB_DMC_TSC10=y
CONFIG_TOUCHSCREEN_USB_IRTOUCH=y
CONFIG_TOUCHSCREEN_USB_IDEALTEK=y
CONFIG_TOUCHSCREEN_USB_GENERAL_TOUCH=y
CONFIG_TOUCHSCREEN_USB_GOTOP=y
CONFIG_TOUCHSCREEN_USB_JASTEC=y
CONFIG_TOUCHSCREEN_USB_E2I=y
CONFIG_TOUCHSCREEN_USB_ZYTRONIC=y
CONFIG_TOUCHSCREEN_USB_ETT_TC45USB=y
CONFIG_TOUCHSCREEN_USB_NEXIO=y
CONFIG_TOUCHSCREEN_TOUCHIT213=y
CONFIG_TOUCHSCREEN_TSC2005=y
CONFIG_TOUCHSCREEN_TSC2007=y
CONFIG_TOUCHSCREEN_PCAP=y
CONFIG_TOUCHSCREEN_ST1232=y
CONFIG_TOUCHSCREEN_STMPE=y
CONFIG_TOUCHSCREEN_TPS6507X=y
CONFIG_INPUT_MISC=y
CONFIG_INPUT_88PM860X_ONKEY=y
CONFIG_INPUT_AB8500_PONKEY=y
CONFIG_INPUT_AD714X=y
CONFIG_INPUT_AD714X_I2C=y
CONFIG_INPUT_AD714X_SPI=y
CONFIG_INPUT_PCSPKR=y
CONFIG_INPUT_MAX8925_ONKEY=y
CONFIG_INPUT_APANEL=y
CONFIG_INPUT_WISTRON_BTNS=y
CONFIG_INPUT_ATLAS_BTNS=y
CONFIG_INPUT_ATI_REMOTE=y
CONFIG_INPUT_ATI_REMOTE2=y
CONFIG_INPUT_KEYSPAN_REMOTE=y
CONFIG_INPUT_POWERMATE=y
CONFIG_INPUT_YEALINK=y
CONFIG_INPUT_CM109=y
CONFIG_INPUT_TWL4030_PWRBUTTON=y
CONFIG_INPUT_TWL4030_VIBRA=y
CONFIG_INPUT_UINPUT=y
CONFIG_INPUT_PCF50633_PMU=m
CONFIG_INPUT_PCF8574=y
CONFIG_INPUT_GPIO_ROTARY_ENCODER=y
CONFIG_INPUT_WM831X_ON=y
CONFIG_INPUT_PCAP=y
CONFIG_INPUT_ADXL34X=y
CONFIG_INPUT_ADXL34X_I2C=y
CONFIG_INPUT_ADXL34X_SPI=y
CONFIG_INPUT_CMA3000=y
CONFIG_INPUT_CMA3000_I2C=y

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=y
CONFIG_SERIO_CT82C710=y
CONFIG_SERIO_PARKBD=y
CONFIG_SERIO_PCIPS2=y
CONFIG_SERIO_LIBPS2=y
CONFIG_SERIO_RAW=y
CONFIG_SERIO_ALTERA_PS2=y
CONFIG_SERIO_PS2MULT=y
CONFIG_GAMEPORT=y
CONFIG_GAMEPORT_NS558=y
CONFIG_GAMEPORT_L4=y
CONFIG_GAMEPORT_EMU10K1=y
CONFIG_GAMEPORT_FM801=y

#
# Character devices
#
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_VT_HW_CONSOLE_BINDING=y
CONFIG_UNIX98_PTYS=y
CONFIG_DEVPTS_MULTIPLE_INSTANCES=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
CONFIG_SERIAL_NONSTANDARD=y
CONFIG_ROCKETPORT=m
CONFIG_CYCLADES=m
CONFIG_CYZ_INTR=y
CONFIG_MOXA_INTELLIO=m
CONFIG_MOXA_SMARTIO=m
CONFIG_SYNCLINK=m
CONFIG_SYNCLINKMP=m
CONFIG_SYNCLINK_GT=m
CONFIG_NOZOMI=m
CONFIG_ISI=m
CONFIG_N_HDLC=m
CONFIG_N_GSM=m
CONFIG_DEVKMEM=y
CONFIG_STALDRV=y

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_PNP=y
CONFIG_SERIAL_8250_CS=m
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_FOURPORT=m
CONFIG_SERIAL_8250_ACCENT=m
CONFIG_SERIAL_8250_BOCA=m
CONFIG_SERIAL_8250_EXAR_ST16C554=m
CONFIG_SERIAL_8250_HUB6=m
CONFIG_SERIAL_8250_SHARE_IRQ=y
CONFIG_SERIAL_8250_DETECT_IRQ=y
CONFIG_SERIAL_8250_RSA=y
CONFIG_SERIAL_8250_MCA=m

#
# Non-8250 serial port support
#
CONFIG_SERIAL_MAX3100=m
CONFIG_SERIAL_MAX3107=m
CONFIG_SERIAL_MRST_MAX3110=m
CONFIG_SERIAL_MFD_HSU=m
CONFIG_SERIAL_UARTLITE=m
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_SERIAL_JSM=m
CONFIG_SERIAL_OF_PLATFORM=m
CONFIG_SERIAL_TIMBERDALE=m
CONFIG_SERIAL_ALTERA_JTAGUART=m
CONFIG_SERIAL_ALTERA_UART=m
CONFIG_SERIAL_ALTERA_UART_MAXPORTS=4
CONFIG_SERIAL_ALTERA_UART_BAUDRATE=115200
CONFIG_SERIAL_IFX6X60=m
CONFIG_SERIAL_PCH_UART=m
CONFIG_TTY_PRINTK=y
CONFIG_PRINTER=y
CONFIG_LP_CONSOLE=y
CONFIG_PPDEV=m
CONFIG_HVC_DRIVER=y
CONFIG_VIRTIO_CONSOLE=y
CONFIG_IPMI_HANDLER=m
# CONFIG_IPMI_PANIC_EVENT is not set
CONFIG_IPMI_DEVICE_INTERFACE=m
CONFIG_IPMI_SI=m
CONFIG_IPMI_WATCHDOG=m
CONFIG_IPMI_POWEROFF=m
CONFIG_HW_RANDOM=y
CONFIG_HW_RANDOM_TIMERIOMEM=m
CONFIG_HW_RANDOM_INTEL=m
CONFIG_HW_RANDOM_AMD=m
CONFIG_HW_RANDOM_GEODE=m
CONFIG_HW_RANDOM_VIA=m
CONFIG_HW_RANDOM_VIRTIO=m
CONFIG_NVRAM=y
CONFIG_DTLK=m
CONFIG_R3964=m
CONFIG_APPLICOM=m
CONFIG_SONYPI=m

#
# PCMCIA character devices
#
CONFIG_SYNCLINK_CS=m
CONFIG_CARDMAN_4000=m
CONFIG_CARDMAN_4040=m
CONFIG_IPWIRELESS=m
CONFIG_MWAVE=m
CONFIG_SCx200_GPIO=m
CONFIG_PC8736x_GPIO=m
CONFIG_NSC_GPIO=m
CONFIG_RAW_DRIVER=m
CONFIG_MAX_RAW_DEVS=256
CONFIG_HPET=y
CONFIG_HPET_MMAP=y
CONFIG_HANGCHECK_TIMER=m
CONFIG_TCG_TPM=m
CONFIG_TCG_TIS=m
CONFIG_TCG_NSC=m
CONFIG_TCG_ATMEL=m
CONFIG_TCG_INFINEON=m
CONFIG_TELCLOCK=m
CONFIG_DEVPORT=y
CONFIG_RAMOOPS=m
CONFIG_I2C=y
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_COMPAT=y
CONFIG_I2C_CHARDEV=y
CONFIG_I2C_MUX=m

#
# Multiplexer I2C Chip support
#
CONFIG_I2C_MUX_GPIO=m
CONFIG_I2C_MUX_PCA9541=m
CONFIG_I2C_MUX_PCA954x=m
CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_SMBUS=m
CONFIG_I2C_ALGOBIT=y
CONFIG_I2C_ALGOPCA=m

#
# I2C Hardware Bus support
#

#
# PC SMBus host controller drivers
#
CONFIG_I2C_ALI1535=m
CONFIG_I2C_ALI1563=m
CONFIG_I2C_ALI15X3=m
CONFIG_I2C_AMD756=m
CONFIG_I2C_AMD756_S4882=m
CONFIG_I2C_AMD8111=m
CONFIG_I2C_I801=m
CONFIG_I2C_ISCH=m
CONFIG_I2C_PIIX4=m
CONFIG_I2C_NFORCE2=m
CONFIG_I2C_NFORCE2_S4985=m
CONFIG_I2C_SIS5595=m
CONFIG_I2C_SIS630=m
CONFIG_I2C_SIS96X=m
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m

#
# ACPI drivers
#
CONFIG_I2C_SCMI=m

#
# I2C system bus drivers (mostly embedded / system-on-chip) #
CONFIG_I2C_GPIO=m
# CONFIG_I2C_INTEL_MID is not set CONFIG_I2C_OCORES=m
CONFIG_I2C_PCA_PLATFORM=m
CONFIG_I2C_PXA=m
CONFIG_I2C_PXA_PCI=y
CONFIG_I2C_SIMTEC=m
CONFIG_I2C_XILINX=m
CONFIG_I2C_EG20T=m

#
# External I2C/SMBus adapter drivers
#
CONFIG_I2C_DIOLAN_U2C=m
CONFIG_I2C_PARPORT=m
CONFIG_I2C_PARPORT_LIGHT=m
CONFIG_I2C_TAOS_EVM=m
CONFIG_I2C_TINY_USB=m

#
# Other I2C/SMBus bus drivers
#
# CONFIG_I2C_PCA_ISA is not set
CONFIG_I2C_STUB=m
CONFIG_SCx200_I2C=m
CONFIG_SCx200_I2C_SCL=12
CONFIG_SCx200_I2C_SDA=13
CONFIG_SCx200_ACB=m
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
CONFIG_SPI=y
CONFIG_SPI_MASTER=y

#
# SPI Master Controller Drivers
#
CONFIG_SPI_ALTERA=m
CONFIG_SPI_BITBANG=m
CONFIG_SPI_BUTTERFLY=m
CONFIG_SPI_GPIO=m
CONFIG_SPI_LM70_LLP=m
CONFIG_SPI_OC_TINY=m
CONFIG_SPI_PXA2XX=m
CONFIG_SPI_PXA2XX_PCI=y
CONFIG_SPI_TOPCLIFF_PCH=m
CONFIG_SPI_XILINX=m
CONFIG_SPI_DESIGNWARE=y
CONFIG_SPI_DW_PCI=m
# CONFIG_SPI_DW_MID_DMA is not set

#
# SPI Protocol Masters
#
CONFIG_SPI_SPIDEV=m
CONFIG_SPI_TLE62X0=m

#
# PPS support
#
CONFIG_PPS=m
# CONFIG_PPS_DEBUG is not set
# CONFIG_NTP_PPS is not set

#
# PPS clients support
#
CONFIG_PPS_CLIENT_KTIMER=m
CONFIG_PPS_CLIENT_LDISC=m
CONFIG_PPS_CLIENT_PARPORT=m

#
# PPS generators support
#
CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
CONFIG_GPIOLIB=y
CONFIG_GPIO_SYSFS=y
CONFIG_GPIO_MAX730X=m

#
# Memory mapped GPIO expanders:
#
CONFIG_GPIO_BASIC_MMIO=m
CONFIG_GPIO_IT8761E=m
CONFIG_GPIO_SCH=m
CONFIG_GPIO_VX855=m

#
# I2C GPIO expanders:
#
CONFIG_GPIO_MAX7300=m
CONFIG_GPIO_MAX732X=m
CONFIG_GPIO_PCA953X=m
CONFIG_GPIO_PCF857X=m
CONFIG_GPIO_SX150X=y
CONFIG_GPIO_STMPE=y
CONFIG_GPIO_TC3589X=y
CONFIG_GPIO_TWL4030=m
CONFIG_GPIO_WM831X=m
CONFIG_GPIO_WM8350=m
CONFIG_GPIO_WM8994=m
CONFIG_GPIO_ADP5520=m
CONFIG_GPIO_ADP5588=m

#
# PCI GPIO expanders:
#
CONFIG_GPIO_CS5535=m
CONFIG_GPIO_LANGWELL=y
CONFIG_GPIO_PCH=m
CONFIG_GPIO_ML_IOH=m
CONFIG_GPIO_TIMBERDALE=y
CONFIG_GPIO_RDC321X=m

#
# SPI GPIO expanders:
#
CONFIG_GPIO_MAX7301=m
CONFIG_GPIO_MCP23S08=m
CONFIG_GPIO_MC33880=m
CONFIG_GPIO_74X164=m

#
# AC97 GPIO expanders:
#
CONFIG_GPIO_UCB1400=y

#
# MODULbus GPIO expanders:
#
CONFIG_GPIO_JANZ_TTL=m
CONFIG_W1=m
CONFIG_W1_CON=y

#
# 1-wire Bus Masters
#
CONFIG_W1_MASTER_MATROX=m
CONFIG_W1_MASTER_DS2490=m
CONFIG_W1_MASTER_DS2482=m
CONFIG_W1_MASTER_GPIO=m

#
# 1-wire Slaves
#
CONFIG_W1_SLAVE_THERM=m
CONFIG_W1_SLAVE_SMEM=m
CONFIG_W1_SLAVE_DS2423=m
CONFIG_W1_SLAVE_DS2431=m
CONFIG_W1_SLAVE_DS2433=m
CONFIG_W1_SLAVE_DS2433_CRC=y
CONFIG_W1_SLAVE_DS2760=m
CONFIG_W1_SLAVE_BQ27000=m
CONFIG_POWER_SUPPLY=y
# CONFIG_POWER_SUPPLY_DEBUG is not set
CONFIG_PDA_POWER=m
CONFIG_MAX8925_POWER=m
CONFIG_WM831X_BACKUP=m
CONFIG_WM831X_POWER=m
CONFIG_WM8350_POWER=m
# CONFIG_TEST_POWER is not set
CONFIG_BATTERY_DS2760=m
CONFIG_BATTERY_DS2782=m
CONFIG_BATTERY_OLPC=m
CONFIG_BATTERY_BQ20Z75=m
CONFIG_BATTERY_BQ27x00=m
CONFIG_BATTERY_BQ27X00_I2C=y
CONFIG_BATTERY_BQ27X00_PLATFORM=y
CONFIG_BATTERY_DA9030=m
CONFIG_BATTERY_MAX17040=m
CONFIG_BATTERY_MAX17042=m
CONFIG_CHARGER_PCF50633=m
CONFIG_CHARGER_ISP1704=m
CONFIG_CHARGER_TWL4030=m
CONFIG_CHARGER_GPIO=m
CONFIG_HWMON=y
CONFIG_HWMON_VID=m
# CONFIG_HWMON_DEBUG_CHIP is not set

#
# Native drivers
#
CONFIG_SENSORS_ABITUGURU=m
CONFIG_SENSORS_ABITUGURU3=m
CONFIG_SENSORS_AD7414=m
CONFIG_SENSORS_AD7418=m
CONFIG_SENSORS_ADCXX=m
CONFIG_SENSORS_ADM1021=m
CONFIG_SENSORS_ADM1025=m
CONFIG_SENSORS_ADM1026=m
CONFIG_SENSORS_ADM1029=m
CONFIG_SENSORS_ADM1031=m
CONFIG_SENSORS_ADM9240=m
CONFIG_SENSORS_ADT7411=m
CONFIG_SENSORS_ADT7462=m
CONFIG_SENSORS_ADT7470=m
CONFIG_SENSORS_ADT7475=m
CONFIG_SENSORS_ASC7621=m
CONFIG_SENSORS_K8TEMP=m
CONFIG_SENSORS_K10TEMP=m
CONFIG_SENSORS_ASB100=m
CONFIG_SENSORS_ATXP1=m
CONFIG_SENSORS_DS620=m
CONFIG_SENSORS_DS1621=m
CONFIG_SENSORS_I5K_AMB=m
CONFIG_SENSORS_F71805F=m
CONFIG_SENSORS_F71882FG=m
CONFIG_SENSORS_F75375S=m
CONFIG_SENSORS_FSCHMD=m
CONFIG_SENSORS_G760A=m
CONFIG_SENSORS_GL518SM=m
CONFIG_SENSORS_GL520SM=m
CONFIG_SENSORS_GPIO_FAN=m
CONFIG_SENSORS_CORETEMP=m
CONFIG_SENSORS_PKGTEMP=m
CONFIG_SENSORS_IBMAEM=m
CONFIG_SENSORS_IBMPEX=m
CONFIG_SENSORS_IT87=m
CONFIG_SENSORS_JC42=m
CONFIG_SENSORS_LINEAGE=m
CONFIG_SENSORS_LM63=m
CONFIG_SENSORS_LM70=m
CONFIG_SENSORS_LM73=m
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM77=m
CONFIG_SENSORS_LM78=m
CONFIG_SENSORS_LM80=m
CONFIG_SENSORS_LM83=m
CONFIG_SENSORS_LM85=m
CONFIG_SENSORS_LM87=m
CONFIG_SENSORS_LM90=m
CONFIG_SENSORS_LM92=m
CONFIG_SENSORS_LM93=m
CONFIG_SENSORS_LTC4151=m
CONFIG_SENSORS_LTC4215=m
CONFIG_SENSORS_LTC4245=m
CONFIG_SENSORS_LTC4261=m
CONFIG_SENSORS_LM95241=m
CONFIG_SENSORS_MAX1111=m
CONFIG_SENSORS_MAX1619=m
CONFIG_SENSORS_MAX6639=m
CONFIG_SENSORS_MAX6650=m
CONFIG_SENSORS_PC87360=m
CONFIG_SENSORS_PC87427=m
CONFIG_SENSORS_PCF8591=m
CONFIG_PMBUS=m
CONFIG_SENSORS_PMBUS=m
CONFIG_SENSORS_MAX16064=m
CONFIG_SENSORS_MAX34440=m
CONFIG_SENSORS_MAX8688=m
CONFIG_SENSORS_SHT15=m
CONFIG_SENSORS_SHT21=m
CONFIG_SENSORS_SIS5595=m
CONFIG_SENSORS_SMM665=m
CONFIG_SENSORS_DME1737=m
CONFIG_SENSORS_EMC1403=m
CONFIG_SENSORS_EMC2103=m
CONFIG_SENSORS_SMSC47M1=m
CONFIG_SENSORS_SMSC47M192=m
CONFIG_SENSORS_SMSC47B397=m
CONFIG_SENSORS_SCH5627=m
CONFIG_SENSORS_ADS1015=m
CONFIG_SENSORS_ADS7828=m
CONFIG_SENSORS_ADS7871=m
CONFIG_SENSORS_AMC6821=m
CONFIG_SENSORS_THMC50=m
CONFIG_SENSORS_TMP102=m
CONFIG_SENSORS_TMP401=m
CONFIG_SENSORS_TMP421=m
# CONFIG_SENSORS_TWL4030_MADC is not set
CONFIG_SENSORS_VIA_CPUTEMP=m
CONFIG_SENSORS_VIA686A=m
CONFIG_SENSORS_VT1211=m
CONFIG_SENSORS_VT8231=m
CONFIG_SENSORS_W83781D=m
CONFIG_SENSORS_W83791D=m
CONFIG_SENSORS_W83792D=m
CONFIG_SENSORS_W83793=m
CONFIG_SENSORS_W83795=m
# CONFIG_SENSORS_W83795_FANCTRL is not set
CONFIG_SENSORS_W83L785TS=m
CONFIG_SENSORS_W83L786NG=m
CONFIG_SENSORS_W83627HF=m
CONFIG_SENSORS_W83627EHF=m
CONFIG_SENSORS_WM831X=m
CONFIG_SENSORS_WM8350=m
CONFIG_SENSORS_APPLESMC=m
CONFIG_SENSORS_MC13783_ADC=m

#
# ACPI drivers
#
CONFIG_SENSORS_ATK0110=m
CONFIG_THERMAL=y
CONFIG_THERMAL_HWMON=y
CONFIG_WATCHDOG=y
# CONFIG_WATCHDOG_NOWAYOUT is not set

#
# Watchdog Device Drivers
#
CONFIG_SOFT_WATCHDOG=m
CONFIG_WM831X_WATCHDOG=m
CONFIG_WM8350_WATCHDOG=m
CONFIG_TWL4030_WATCHDOG=m
CONFIG_ACQUIRE_WDT=m
CONFIG_ADVANTECH_WDT=m
CONFIG_ALIM1535_WDT=m
CONFIG_ALIM7101_WDT=m
CONFIG_F71808E_WDT=m
CONFIG_SP5100_TCO=m
CONFIG_GEODE_WDT=m
CONFIG_SC520_WDT=m
CONFIG_SBC_FITPC2_WATCHDOG=m
# CONFIG_EUROTECH_WDT is not set
CONFIG_IB700_WDT=m
CONFIG_IBMASR=m
CONFIG_WAFER_WDT=m
CONFIG_I6300ESB_WDT=m
CONFIG_ITCO_WDT=m
CONFIG_ITCO_VENDOR_SUPPORT=y
CONFIG_IT8712F_WDT=m
CONFIG_IT87_WDT=m
CONFIG_HP_WATCHDOG=m
CONFIG_HPWDT_NMI_DECODING=y
CONFIG_SC1200_WDT=m
CONFIG_SCx200_WDT=m
CONFIG_PC87413_WDT=m
CONFIG_NV_TCO=m
CONFIG_RDC321X_WDT=m
CONFIG_60XX_WDT=m
CONFIG_SBC8360_WDT=m
CONFIG_SBC7240_WDT=m
CONFIG_CPU5_WDT=m
CONFIG_SMSC_SCH311X_WDT=m
CONFIG_SMSC37B787_WDT=m
CONFIG_W83627HF_WDT=m
CONFIG_W83697HF_WDT=m
CONFIG_W83697UG_WDT=m
CONFIG_W83877F_WDT=m
CONFIG_W83977F_WDT=m
CONFIG_MACHZ_WDT=m
CONFIG_SBC_EPX_C3_WATCHDOG=m

#
# ISA-based Watchdog Cards
#
CONFIG_PCWATCHDOG=m
CONFIG_MIXCOMWD=m
# CONFIG_WDT is not set

#
# PCI-based Watchdog Cards
#
CONFIG_PCIPCWATCHDOG=m
CONFIG_WDTPCI=m

#
# USB-based Watchdog Cards
#
CONFIG_USBPCWATCHDOG=m
CONFIG_SSB_POSSIBLE=y

#
# Sonics Silicon Backplane
#
CONFIG_SSB=m
CONFIG_SSB_SPROM=y
CONFIG_SSB_BLOCKIO=y
CONFIG_SSB_PCIHOST_POSSIBLE=y
CONFIG_SSB_PCIHOST=y
CONFIG_SSB_B43_PCI_BRIDGE=y
CONFIG_SSB_PCMCIAHOST_POSSIBLE=y
CONFIG_SSB_PCMCIAHOST=y
CONFIG_SSB_SDIOHOST_POSSIBLE=y
CONFIG_SSB_SDIOHOST=y
# CONFIG_SSB_SILENT is not set
# CONFIG_SSB_DEBUG is not set
CONFIG_SSB_DRIVER_PCICORE_POSSIBLE=y
CONFIG_SSB_DRIVER_PCICORE=y
CONFIG_MFD_SUPPORT=y
CONFIG_MFD_CORE=y
CONFIG_MFD_88PM860X=y
CONFIG_MFD_SM501=m
CONFIG_MFD_SM501_GPIO=y
CONFIG_HTC_PASIC3=m
CONFIG_HTC_I2CPLD=y
CONFIG_UCB1400_CORE=m
CONFIG_TPS6105X=m
CONFIG_TPS65010=m
CONFIG_TPS6507X=m
CONFIG_TWL4030_CORE=y
CONFIG_TWL4030_MADC=m
CONFIG_TWL4030_CODEC=y
CONFIG_TWL6030_PWM=m
CONFIG_MFD_STMPE=y
CONFIG_MFD_TC3589X=y
# CONFIG_MFD_TMIO is not set
CONFIG_PMIC_DA903X=y
CONFIG_PMIC_ADP5520=y
CONFIG_MFD_MAX8925=y
CONFIG_MFD_MAX8997=y
CONFIG_MFD_MAX8998=y
CONFIG_MFD_WM8400=m
CONFIG_MFD_WM831X=y
CONFIG_MFD_WM831X_I2C=y
CONFIG_MFD_WM831X_SPI=y
CONFIG_MFD_WM8350=y
CONFIG_MFD_WM8350_I2C=y
CONFIG_MFD_WM8994=y
CONFIG_MFD_PCF50633=m
CONFIG_MFD_MC13783=m
CONFIG_MFD_MC13XXX=m
CONFIG_PCF50633_ADC=m
CONFIG_PCF50633_GPIO=m
CONFIG_ABX500_CORE=y
CONFIG_AB3100_CORE=y
CONFIG_AB3100_OTP=m
CONFIG_EZX_PCAP=y
CONFIG_AB8500_CORE=y
CONFIG_AB8500_DEBUG=y
CONFIG_AB8500_GPADC=y
CONFIG_AB3550_CORE=y
CONFIG_MFD_CS5535=m
CONFIG_MFD_TIMBERDALE=y
CONFIG_LPC_SCH=m
CONFIG_MFD_RDC321X=m
CONFIG_MFD_JANZ_CMODIO=m
CONFIG_MFD_TPS6586X=y
CONFIG_MFD_VX855=m
CONFIG_MFD_WL1273_CORE=m
CONFIG_REGULATOR=y
# CONFIG_REGULATOR_DEBUG is not set
CONFIG_REGULATOR_DUMMY=y
CONFIG_REGULATOR_FIXED_VOLTAGE=m
CONFIG_REGULATOR_VIRTUAL_CONSUMER=m
CONFIG_REGULATOR_USERSPACE_CONSUMER=m
CONFIG_REGULATOR_BQ24022=m
CONFIG_REGULATOR_MAX1586=m
CONFIG_REGULATOR_MAX8649=m
CONFIG_REGULATOR_MAX8660=m
CONFIG_REGULATOR_MAX8925=m
CONFIG_REGULATOR_MAX8952=m
CONFIG_REGULATOR_MAX8997=m
CONFIG_REGULATOR_MAX8998=m
CONFIG_REGULATOR_TWL4030=y
CONFIG_REGULATOR_WM831X=m
CONFIG_REGULATOR_WM8350=m
CONFIG_REGULATOR_WM8400=m
CONFIG_REGULATOR_WM8994=m
CONFIG_REGULATOR_DA903X=m
CONFIG_REGULATOR_PCF50633=m
CONFIG_REGULATOR_LP3971=m
CONFIG_REGULATOR_LP3972=m
CONFIG_REGULATOR_PCAP=m
CONFIG_REGULATOR_MC13XXX_CORE=m
CONFIG_REGULATOR_MC13783=m
CONFIG_REGULATOR_MC13892=m
CONFIG_REGULATOR_AB3100=m
CONFIG_REGULATOR_TPS6105X=m
CONFIG_REGULATOR_TPS65023=m
CONFIG_REGULATOR_TPS6507X=m
CONFIG_REGULATOR_88PM8607=y
CONFIG_REGULATOR_ISL6271A=m
CONFIG_REGULATOR_AD5398=m
CONFIG_REGULATOR_AB8500=y
CONFIG_REGULATOR_TPS6586X=m
CONFIG_REGULATOR_TPS6524X=m
CONFIG_MEDIA_SUPPORT=y

#
# Multimedia core support
#
CONFIG_MEDIA_CONTROLLER=y
CONFIG_VIDEO_DEV=y
CONFIG_VIDEO_V4L2_COMMON=y
CONFIG_VIDEO_V4L2_SUBDEV_API=y
CONFIG_DVB_CORE=y
CONFIG_VIDEO_MEDIA=y

#
# Multimedia drivers
#
CONFIG_VIDEO_SAA7146=m
CONFIG_VIDEO_SAA7146_VV=m
CONFIG_RC_CORE=y
CONFIG_LIRC=y
CONFIG_RC_MAP=y
CONFIG_IR_NEC_DECODER=y
CONFIG_IR_RC5_DECODER=y
CONFIG_IR_RC6_DECODER=y
CONFIG_IR_JVC_DECODER=y
CONFIG_IR_SONY_DECODER=y
CONFIG_IR_RC5_SZ_DECODER=y
CONFIG_IR_LIRC_CODEC=y
CONFIG_IR_ENE=m
CONFIG_IR_IMON=m
CONFIG_IR_MCEUSB=m
CONFIG_IR_ITE_CIR=m
CONFIG_IR_NUVOTON=m
CONFIG_IR_STREAMZAP=m
CONFIG_IR_WINBOND_CIR=m
# CONFIG_RC_LOOPBACK is not set
CONFIG_MEDIA_ATTACH=y
CONFIG_MEDIA_TUNER=y
CONFIG_MEDIA_TUNER_CUSTOMISE=y

#
# Customize TV tuners
#
CONFIG_MEDIA_TUNER_SIMPLE=m
CONFIG_MEDIA_TUNER_TDA8290=m
CONFIG_MEDIA_TUNER_TDA827X=m
CONFIG_MEDIA_TUNER_TDA18271=m
CONFIG_MEDIA_TUNER_TDA9887=m
CONFIG_MEDIA_TUNER_TEA5761=m
CONFIG_MEDIA_TUNER_TEA5767=m
CONFIG_MEDIA_TUNER_MT20XX=m
CONFIG_MEDIA_TUNER_MT2060=m
CONFIG_MEDIA_TUNER_MT2266=m
CONFIG_MEDIA_TUNER_MT2131=m
CONFIG_MEDIA_TUNER_QT1010=m
CONFIG_MEDIA_TUNER_XC2028=m
CONFIG_MEDIA_TUNER_XC5000=m
CONFIG_MEDIA_TUNER_MXL5005S=m
CONFIG_MEDIA_TUNER_MXL5007T=m
CONFIG_MEDIA_TUNER_MC44S803=m
CONFIG_MEDIA_TUNER_MAX2165=m
CONFIG_MEDIA_TUNER_TDA18218=m
CONFIG_VIDEO_V4L2=y
CONFIG_VIDEOBUF_GEN=m
CONFIG_VIDEOBUF_DMA_SG=m
CONFIG_VIDEOBUF_VMALLOC=m
CONFIG_VIDEOBUF_DMA_CONTIG=m
CONFIG_VIDEOBUF_DVB=m
CONFIG_VIDEO_BTCX=m
CONFIG_VIDEO_TVEEPROM=m
CONFIG_VIDEO_TUNER=m
CONFIG_VIDEOBUF2_CORE=m
CONFIG_VIDEO_CAPTURE_DRIVERS=y
# CONFIG_VIDEO_ADV_DEBUG is not set
# CONFIG_VIDEO_FIXED_MINOR_RANGES is not set
CONFIG_VIDEO_HELPER_CHIPS_AUTO=y
CONFIG_VIDEO_IR_I2C=y

#
# Audio decoders
#
CONFIG_VIDEO_TVAUDIO=m
CONFIG_VIDEO_TDA7432=m
CONFIG_VIDEO_TDA9840=m
CONFIG_VIDEO_TEA6415C=m
CONFIG_VIDEO_TEA6420=m
CONFIG_VIDEO_MSP3400=m
CONFIG_VIDEO_CS5345=m
CONFIG_VIDEO_CS53L32A=m
CONFIG_VIDEO_M52790=m
CONFIG_VIDEO_WM8775=m
CONFIG_VIDEO_WM8739=m
CONFIG_VIDEO_VP27SMPX=m

#
# RDS decoders
#
CONFIG_VIDEO_SAA6588=m

#
# Video decoders
#
CONFIG_VIDEO_ADV7180=m
CONFIG_VIDEO_BT819=m
CONFIG_VIDEO_BT856=m
CONFIG_VIDEO_BT866=m
CONFIG_VIDEO_KS0127=m
CONFIG_VIDEO_OV7670=m
CONFIG_VIDEO_MT9V011=m
CONFIG_VIDEO_SAA7110=m
CONFIG_VIDEO_SAA711X=m
CONFIG_VIDEO_SAA717X=m
CONFIG_VIDEO_TVP5150=m
CONFIG_VIDEO_VPX3220=m

#
# Video and audio decoders
#
CONFIG_VIDEO_CX25840=m

#
# MPEG video encoders
#
CONFIG_VIDEO_CX2341X=m

#
# Video encoders
#
CONFIG_VIDEO_SAA7127=m
CONFIG_VIDEO_SAA7185=m
CONFIG_VIDEO_ADV7170=m
CONFIG_VIDEO_ADV7175=m

#
# Video improvement chips
#
CONFIG_VIDEO_UPD64031A=m
CONFIG_VIDEO_UPD64083=m
CONFIG_VIDEO_BT848=m
CONFIG_VIDEO_BT848_DVB=y
CONFIG_VIDEO_PMS=m
CONFIG_VIDEO_BWQCAM=m
CONFIG_VIDEO_CQCAM=m
CONFIG_VIDEO_W9966=m
CONFIG_VIDEO_CPIA2=m
CONFIG_VIDEO_ZORAN=m
CONFIG_VIDEO_ZORAN_DC30=m
CONFIG_VIDEO_ZORAN_ZR36060=m
CONFIG_VIDEO_ZORAN_BUZ=m
CONFIG_VIDEO_ZORAN_DC10=m
CONFIG_VIDEO_ZORAN_LML33=m
CONFIG_VIDEO_ZORAN_LML33R10=m
CONFIG_VIDEO_ZORAN_AVS6EYES=m
CONFIG_VIDEO_MEYE=m
CONFIG_VIDEO_SAA7134=m
CONFIG_VIDEO_SAA7134_ALSA=m
CONFIG_VIDEO_SAA7134_RC=y
CONFIG_VIDEO_SAA7134_DVB=m
CONFIG_VIDEO_MXB=m
CONFIG_VIDEO_HEXIUM_ORION=m
CONFIG_VIDEO_HEXIUM_GEMINI=m
CONFIG_VIDEO_TIMBERDALE=m
CONFIG_VIDEO_CX88=m
CONFIG_VIDEO_CX88_ALSA=m
CONFIG_VIDEO_CX88_BLACKBIRD=m
CONFIG_VIDEO_CX88_DVB=m
CONFIG_VIDEO_CX88_MPEG=m
CONFIG_VIDEO_CX88_VP3054=m
CONFIG_VIDEO_CX23885=m
CONFIG_MEDIA_ALTERA_CI=m
CONFIG_VIDEO_AU0828=m
CONFIG_VIDEO_IVTV=m
CONFIG_VIDEO_FB_IVTV=m
CONFIG_VIDEO_CX18=m
CONFIG_VIDEO_CX18_ALSA=m
CONFIG_VIDEO_SAA7164=m
CONFIG_VIDEO_CAFE_CCIC=m
CONFIG_VIDEO_SR030PC30=m
CONFIG_VIDEO_VIA_CAMERA=m
CONFIG_VIDEO_NOON010PC30=m
CONFIG_SOC_CAMERA=m
CONFIG_SOC_CAMERA_IMX074=m
CONFIG_SOC_CAMERA_MT9M001=m
CONFIG_SOC_CAMERA_MT9M111=m
CONFIG_SOC_CAMERA_MT9T031=m
CONFIG_SOC_CAMERA_MT9T112=m
CONFIG_SOC_CAMERA_MT9V022=m
CONFIG_SOC_CAMERA_RJ54N1=m
CONFIG_SOC_CAMERA_TW9910=m
CONFIG_SOC_CAMERA_PLATFORM=m
CONFIG_SOC_CAMERA_OV2640=m
CONFIG_SOC_CAMERA_OV6650=m
CONFIG_SOC_CAMERA_OV772X=m
CONFIG_SOC_CAMERA_OV9640=m
CONFIG_SOC_CAMERA_OV9740=m
CONFIG_V4L_USB_DRIVERS=y
CONFIG_USB_VIDEO_CLASS=m
CONFIG_USB_VIDEO_CLASS_INPUT_EVDEV=y
CONFIG_USB_GSPCA=m
CONFIG_USB_M5602=m
CONFIG_USB_STV06XX=m
CONFIG_USB_GL860=m
CONFIG_USB_GSPCA_BENQ=m
CONFIG_USB_GSPCA_CONEX=m
CONFIG_USB_GSPCA_CPIA1=m
CONFIG_USB_GSPCA_ETOMS=m
CONFIG_USB_GSPCA_FINEPIX=m
CONFIG_USB_GSPCA_JEILINJ=m
CONFIG_USB_GSPCA_KONICA=m
CONFIG_USB_GSPCA_MARS=m
CONFIG_USB_GSPCA_MR97310A=m
CONFIG_USB_GSPCA_NW80X=m
CONFIG_USB_GSPCA_OV519=m
CONFIG_USB_GSPCA_OV534=m
CONFIG_USB_GSPCA_OV534_9=m
CONFIG_USB_GSPCA_PAC207=m
CONFIG_USB_GSPCA_PAC7302=m
CONFIG_USB_GSPCA_PAC7311=m
CONFIG_USB_GSPCA_SN9C2028=m
CONFIG_USB_GSPCA_SN9C20X=m
CONFIG_USB_GSPCA_SONIXB=m
CONFIG_USB_GSPCA_SONIXJ=m
CONFIG_USB_GSPCA_SPCA500=m
CONFIG_USB_GSPCA_SPCA501=m
CONFIG_USB_GSPCA_SPCA505=m
CONFIG_USB_GSPCA_SPCA506=m
CONFIG_USB_GSPCA_SPCA508=m
CONFIG_USB_GSPCA_SPCA561=m
CONFIG_USB_GSPCA_SPCA1528=m
CONFIG_USB_GSPCA_SQ905=m
CONFIG_USB_GSPCA_SQ905C=m
CONFIG_USB_GSPCA_SQ930X=m
CONFIG_USB_GSPCA_STK014=m
CONFIG_USB_GSPCA_STV0680=m
CONFIG_USB_GSPCA_SUNPLUS=m
CONFIG_USB_GSPCA_T613=m
CONFIG_USB_GSPCA_TV8532=m
CONFIG_USB_GSPCA_VC032X=m
CONFIG_USB_GSPCA_VICAM=m
CONFIG_USB_GSPCA_XIRLINK_CIT=m
CONFIG_USB_GSPCA_ZC3XX=m
CONFIG_VIDEO_PVRUSB2=m
CONFIG_VIDEO_PVRUSB2_SYSFS=y
CONFIG_VIDEO_PVRUSB2_DVB=y
# CONFIG_VIDEO_PVRUSB2_DEBUGIFC is not set
CONFIG_VIDEO_HDPVR=m
CONFIG_VIDEO_EM28XX=m
CONFIG_VIDEO_EM28XX_ALSA=m
CONFIG_VIDEO_EM28XX_DVB=m
CONFIG_VIDEO_TLG2300=m
CONFIG_VIDEO_CX231XX=m
CONFIG_VIDEO_CX231XX_RC=y
CONFIG_VIDEO_CX231XX_ALSA=m
CONFIG_VIDEO_CX231XX_DVB=m
CONFIG_VIDEO_USBVISION=m
CONFIG_USB_ET61X251=m
CONFIG_USB_SN9C102=m
CONFIG_USB_PWC=m
# CONFIG_USB_PWC_DEBUG is not set
CONFIG_USB_PWC_INPUT_EVDEV=y
CONFIG_USB_ZR364XX=m
CONFIG_USB_STKWEBCAM=m
CONFIG_USB_S2255=m
CONFIG_V4L_MEM2MEM_DRIVERS=y
# CONFIG_VIDEO_MEM2MEM_TESTDEV is not set
CONFIG_RADIO_ADAPTERS=y
CONFIG_RADIO_CADET=m
CONFIG_RADIO_RTRACK=m
CONFIG_RADIO_RTRACK2=m
CONFIG_RADIO_AZTECH=m
CONFIG_RADIO_GEMTEK=m
CONFIG_RADIO_MAXIRADIO=m
CONFIG_RADIO_MAESTRO=m
CONFIG_RADIO_MIROPCM20=m
CONFIG_RADIO_SF16FMI=m
CONFIG_RADIO_SF16FMR2=m
CONFIG_RADIO_TERRATEC=m
CONFIG_RADIO_TRUST=m
CONFIG_RADIO_TYPHOON=m
CONFIG_RADIO_ZOLTRIX=m
CONFIG_I2C_SI4713=m
CONFIG_RADIO_SI4713=m
CONFIG_USB_DSBR=m
CONFIG_RADIO_SI470X=y
CONFIG_USB_SI470X=m
CONFIG_I2C_SI470X=m
CONFIG_USB_MR800=m
CONFIG_RADIO_TEA5764=m
CONFIG_RADIO_SAA7706H=m
CONFIG_RADIO_TEF6862=m
CONFIG_RADIO_TIMBERDALE=m
CONFIG_RADIO_WL1273=m

#
# Texas Instruments WL128x FM driver (ST based)
#
CONFIG_RADIO_WL128X=m
CONFIG_DVB_MAX_ADAPTERS=8
CONFIG_DVB_DYNAMIC_MINORS=y
CONFIG_DVB_CAPTURE_DRIVERS=y

#
# Supported SAA7146 based PCI Adapters
#
CONFIG_TTPCI_EEPROM=m
CONFIG_DVB_AV7110=m
CONFIG_DVB_AV7110_OSD=y
CONFIG_DVB_BUDGET_CORE=m
CONFIG_DVB_BUDGET=m
CONFIG_DVB_BUDGET_CI=m
CONFIG_DVB_BUDGET_AV=m
CONFIG_DVB_BUDGET_PATCH=m

#
# Supported USB Adapters
#
CONFIG_DVB_USB=m
# CONFIG_DVB_USB_DEBUG is not set
CONFIG_DVB_USB_A800=m
CONFIG_DVB_USB_DIBUSB_MB=m
CONFIG_DVB_USB_DIBUSB_MB_FAULTY=y
CONFIG_DVB_USB_DIBUSB_MC=m
CONFIG_DVB_USB_DIB0700=m
CONFIG_DVB_USB_UMT_010=m
CONFIG_DVB_USB_CXUSB=m
CONFIG_DVB_USB_M920X=m
CONFIG_DVB_USB_GL861=m
CONFIG_DVB_USB_AU6610=m
CONFIG_DVB_USB_DIGITV=m
CONFIG_DVB_USB_VP7045=m
CONFIG_DVB_USB_VP702X=m
CONFIG_DVB_USB_GP8PSK=m
CONFIG_DVB_USB_NOVA_T_USB2=m
CONFIG_DVB_USB_TTUSB2=m
CONFIG_DVB_USB_DTT200U=m
CONFIG_DVB_USB_OPERA1=m
CONFIG_DVB_USB_AF9005=m
CONFIG_DVB_USB_AF9005_REMOTE=m
CONFIG_DVB_USB_DW2102=m
CONFIG_DVB_USB_CINERGY_T2=m
CONFIG_DVB_USB_ANYSEE=m
CONFIG_DVB_USB_DTV5100=m
CONFIG_DVB_USB_AF9015=m
CONFIG_DVB_USB_CE6230=m
CONFIG_DVB_USB_FRIIO=m
CONFIG_DVB_USB_EC168=m
CONFIG_DVB_USB_AZ6027=m
CONFIG_DVB_USB_LME2510=m
CONFIG_DVB_USB_TECHNISAT_USB2=m
CONFIG_DVB_TTUSB_BUDGET=m
CONFIG_DVB_TTUSB_DEC=m
CONFIG_SMS_SIANO_MDTV=m

#
# Siano module components
#
CONFIG_SMS_USB_DRV=m
CONFIG_SMS_SDIO_DRV=m

#
# Supported FlexCopII (B2C2) Adapters
#
CONFIG_DVB_B2C2_FLEXCOP=m
CONFIG_DVB_B2C2_FLEXCOP_PCI=m
CONFIG_DVB_B2C2_FLEXCOP_USB=m
# CONFIG_DVB_B2C2_FLEXCOP_DEBUG is not set

#
# Supported BT878 Adapters
#
CONFIG_DVB_BT8XX=m

#
# Supported Pluto2 Adapters
#
CONFIG_DVB_PLUTO2=m

#
# Supported SDMC DM1105 Adapters
#
CONFIG_DVB_DM1105=m

#
# Supported FireWire (IEEE 1394) Adapters
#
CONFIG_DVB_FIREDTV=m
CONFIG_DVB_FIREDTV_INPUT=y

#
# Supported Earthsoft PT1 Adapters
#
CONFIG_DVB_PT1=m

#
# Supported Mantis Adapters
#
CONFIG_MANTIS_CORE=m
CONFIG_DVB_MANTIS=m
CONFIG_DVB_HOPPER=m

#
# Supported nGene Adapters
#
CONFIG_DVB_NGENE=m

#
# Supported DVB Frontends
#
CONFIG_DVB_FE_CUSTOMISE=y

#
# Customise DVB Frontends
#

#
# Multistandard (satellite) frontends
#
CONFIG_DVB_STB0899=m
CONFIG_DVB_STB6100=m
CONFIG_DVB_STV090x=m
CONFIG_DVB_STV6110x=m

#
# DVB-S (satellite) frontends
#
CONFIG_DVB_CX24110=m
CONFIG_DVB_CX24123=m
CONFIG_DVB_MT312=m
CONFIG_DVB_ZL10036=m
CONFIG_DVB_ZL10039=m
CONFIG_DVB_S5H1420=m
CONFIG_DVB_STV0288=m
CONFIG_DVB_STB6000=m
CONFIG_DVB_STV0299=m
CONFIG_DVB_STV6110=m
CONFIG_DVB_STV0900=m
CONFIG_DVB_TDA8083=m
CONFIG_DVB_TDA10086=m
CONFIG_DVB_TDA8261=m
CONFIG_DVB_VES1X93=m
CONFIG_DVB_TUNER_ITD1000=m
CONFIG_DVB_TUNER_CX24113=m
CONFIG_DVB_TDA826X=m
CONFIG_DVB_TUA6100=m
CONFIG_DVB_CX24116=m
CONFIG_DVB_SI21XX=m
CONFIG_DVB_DS3000=m
CONFIG_DVB_MB86A16=m

#
# DVB-T (terrestrial) frontends
#
CONFIG_DVB_SP8870=m
CONFIG_DVB_SP887X=m
CONFIG_DVB_CX22700=m
CONFIG_DVB_CX22702=m
CONFIG_DVB_S5H1432=m
CONFIG_DVB_DRX397XD=m
CONFIG_DVB_L64781=m
CONFIG_DVB_TDA1004X=m
CONFIG_DVB_NXT6000=m
CONFIG_DVB_MT352=m
CONFIG_DVB_ZL10353=m
CONFIG_DVB_DIB3000MB=m
CONFIG_DVB_DIB3000MC=m
CONFIG_DVB_DIB7000M=m
CONFIG_DVB_DIB7000P=m
CONFIG_DVB_DIB9000=m
CONFIG_DVB_TDA10048=m
CONFIG_DVB_AF9013=m
CONFIG_DVB_EC100=m
CONFIG_DVB_STV0367=m

#
# DVB-C (cable) frontends
#
CONFIG_DVB_VES1820=m
CONFIG_DVB_TDA10021=m
CONFIG_DVB_TDA10023=m
CONFIG_DVB_STV0297=m

#
# ATSC (North American/Korean Terrestrial/Cable DTV) frontends #
CONFIG_DVB_NXT200X=m
CONFIG_DVB_OR51211=m
CONFIG_DVB_OR51132=m
CONFIG_DVB_BCM3510=m
CONFIG_DVB_LGDT330X=m
CONFIG_DVB_LGDT3305=m
CONFIG_DVB_S5H1409=m
CONFIG_DVB_AU8522=m
CONFIG_DVB_S5H1411=m

#
# ISDB-T (terrestrial) frontends
#
CONFIG_DVB_S921=m
CONFIG_DVB_DIB8000=m
CONFIG_DVB_MB86A20S=m

#
# Digital terrestrial only tuners/PLL
#
CONFIG_DVB_PLL=m
CONFIG_DVB_TUNER_DIB0070=m
CONFIG_DVB_TUNER_DIB0090=m

#
# SEC control devices for DVB-S
#
CONFIG_DVB_LNBP21=m
CONFIG_DVB_ISL6405=m
CONFIG_DVB_ISL6421=m
CONFIG_DVB_ISL6423=m
CONFIG_DVB_LGS8GL5=m
CONFIG_DVB_LGS8GXX=m
CONFIG_DVB_ATBM8830=m
CONFIG_DVB_TDA665x=m
CONFIG_DVB_IX2505V=m

#
# Tools to develop new frontends
#
# CONFIG_DVB_DUMMY_FE is not set

#
# Graphics support
#
CONFIG_AGP=y
CONFIG_AGP_ALI=y
CONFIG_AGP_ATI=y
CONFIG_AGP_AMD=y
CONFIG_AGP_AMD64=y
CONFIG_AGP_INTEL=y
CONFIG_AGP_NVIDIA=y
CONFIG_AGP_SIS=y
CONFIG_AGP_SWORKS=y
CONFIG_AGP_VIA=y
CONFIG_AGP_EFFICEON=y
CONFIG_VGA_ARB=y
CONFIG_VGA_ARB_MAX_GPUS=16
CONFIG_VGA_SWITCHEROO=y
CONFIG_DRM=y
CONFIG_DRM_KMS_HELPER=y
CONFIG_DRM_TTM=y
CONFIG_DRM_TDFX=y
CONFIG_DRM_R128=y
CONFIG_DRM_RADEON=y
# CONFIG_DRM_RADEON_KMS is not set
CONFIG_DRM_I810=y
CONFIG_DRM_I915=m
CONFIG_DRM_I915_KMS=y
CONFIG_DRM_MGA=y
CONFIG_DRM_SIS=m
CONFIG_DRM_VIA=y
CONFIG_DRM_SAVAGE=y
CONFIG_STUB_POULSBO=m
CONFIG_VGASTATE=y
CONFIG_VIDEO_OUTPUT_CONTROL=y
CONFIG_FB=y
CONFIG_FIRMWARE_EDID=y
CONFIG_FB_DDC=y
CONFIG_FB_BOOT_VESA_SUPPORT=y
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
CONFIG_FB_SYS_FILLRECT=y
CONFIG_FB_SYS_COPYAREA=y
CONFIG_FB_SYS_IMAGEBLIT=y
CONFIG_FB_FOREIGN_ENDIAN=y
CONFIG_FB_BOTH_ENDIAN=y
# CONFIG_FB_BIG_ENDIAN is not set
# CONFIG_FB_LITTLE_ENDIAN is not set
CONFIG_FB_SYS_FOPS=y
# CONFIG_FB_WMT_GE_ROPS is not set
CONFIG_FB_DEFERRED_IO=y
CONFIG_FB_HECUBA=y
CONFIG_FB_SVGALIB=y
# CONFIG_FB_MACMODES is not set
CONFIG_FB_BACKLIGHT=y
CONFIG_FB_MODE_HELPERS=y
CONFIG_FB_TILEBLITTING=y

#
# Frame buffer hardware drivers
#
# CONFIG_FB_CIRRUS is not set
CONFIG_FB_PM2=y
CONFIG_FB_PM2_FIFO_DISCONNECT=y
CONFIG_FB_CYBER2000=y
CONFIG_FB_CYBER2000_DDC=y
CONFIG_FB_ARC=y
# CONFIG_FB_ASILIANT is not set
CONFIG_FB_IMSTT=y
# CONFIG_FB_VGA16 is not set
CONFIG_FB_UVESA=y
# CONFIG_FB_VESA is not set
CONFIG_FB_EFI=y
CONFIG_FB_N411=y
CONFIG_FB_HGA=y
CONFIG_FB_S1D13XXX=y
CONFIG_FB_NVIDIA=y
CONFIG_FB_NVIDIA_I2C=y
# CONFIG_FB_NVIDIA_DEBUG is not set
CONFIG_FB_NVIDIA_BACKLIGHT=y
CONFIG_FB_RIVA=y
CONFIG_FB_RIVA_I2C=y
# CONFIG_FB_RIVA_DEBUG is not set
CONFIG_FB_RIVA_BACKLIGHT=y
CONFIG_FB_I810=y
CONFIG_FB_I810_GTF=y
CONFIG_FB_I810_I2C=y
CONFIG_FB_LE80578=y
CONFIG_FB_CARILLO_RANCH=y
CONFIG_FB_INTEL=m
# CONFIG_FB_INTEL_DEBUG is not set
CONFIG_FB_INTEL_I2C=y
CONFIG_FB_MATROX=y
CONFIG_FB_MATROX_MILLENIUM=y
CONFIG_FB_MATROX_MYSTIQUE=y
CONFIG_FB_MATROX_G=y
CONFIG_FB_MATROX_I2C=y
CONFIG_FB_MATROX_MAVEN=y
# CONFIG_FB_RADEON is not set
CONFIG_FB_ATY128=y
CONFIG_FB_ATY128_BACKLIGHT=y
CONFIG_FB_ATY=y
CONFIG_FB_ATY_CT=y
CONFIG_FB_ATY_GENERIC_LCD=y
CONFIG_FB_ATY_GX=y
CONFIG_FB_ATY_BACKLIGHT=y
CONFIG_FB_S3=y
CONFIG_FB_SAVAGE=y
CONFIG_FB_SAVAGE_I2C=y
CONFIG_FB_SAVAGE_ACCEL=y
CONFIG_FB_SIS=y
CONFIG_FB_SIS_300=y
CONFIG_FB_SIS_315=y
CONFIG_FB_VIA=y
# CONFIG_FB_VIA_DIRECT_PROCFS is not set
CONFIG_FB_NEOMAGIC=y
CONFIG_FB_KYRO=y
CONFIG_FB_3DFX=y
CONFIG_FB_3DFX_ACCEL=y
CONFIG_FB_3DFX_I2C=y
CONFIG_FB_VOODOO1=y
CONFIG_FB_VT8623=y
CONFIG_FB_TRIDENT=y
CONFIG_FB_ARK=y
CONFIG_FB_PM3=y
CONFIG_FB_CARMINE=y
# CONFIG_FB_CARMINE_DRAM_EVAL is not set
CONFIG_CARMINE_DRAM_CUSTOM=y
CONFIG_FB_GEODE=y
CONFIG_FB_GEODE_LX=y
CONFIG_FB_GEODE_GX=y
CONFIG_FB_GEODE_GX1=y
CONFIG_FB_TMIO=m
CONFIG_FB_TMIO_ACCELL=y
CONFIG_FB_SM501=m
CONFIG_FB_UDL=m
# CONFIG_FB_VIRTUAL is not set
CONFIG_FB_METRONOME=y
CONFIG_FB_MB862XX=y
CONFIG_FB_MB862XX_PCI_GDC=y
CONFIG_FB_BROADSHEET=y
CONFIG_BACKLIGHT_LCD_SUPPORT=y
CONFIG_LCD_CLASS_DEVICE=m
CONFIG_LCD_L4F00242T03=m
CONFIG_LCD_LMS283GF05=m
CONFIG_LCD_LTV350QV=m
CONFIG_LCD_ILI9320=m
CONFIG_LCD_TDO24M=m
CONFIG_LCD_VGG2432A4=m
CONFIG_LCD_PLATFORM=m
CONFIG_LCD_S6E63M0=m
CONFIG_LCD_LD9040=m
CONFIG_BACKLIGHT_CLASS_DEVICE=y
CONFIG_BACKLIGHT_GENERIC=y
CONFIG_BACKLIGHT_PROGEAR=m
CONFIG_BACKLIGHT_CARILLO_RANCH=m
CONFIG_BACKLIGHT_DA903X=m
CONFIG_BACKLIGHT_MAX8925=m
CONFIG_BACKLIGHT_APPLE=m
CONFIG_BACKLIGHT_SAHARA=m
CONFIG_BACKLIGHT_WM831X=m
CONFIG_BACKLIGHT_ADP5520=m
CONFIG_BACKLIGHT_ADP8860=m
CONFIG_BACKLIGHT_88PM860X=m
CONFIG_BACKLIGHT_PCF50633=m

#
# Display device support
#
CONFIG_DISPLAY_SUPPORT=y

#
# Display hardware drivers
#

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_VGACON_SOFT_SCROLLBACK=y
CONFIG_VGACON_SOFT_SCROLLBACK_SIZE=256
# CONFIG_MDA_CONSOLE is not set
CONFIG_DUMMY_CONSOLE=y
# CONFIG_FRAMEBUFFER_CONSOLE is not set
CONFIG_FONT_8x16=y
CONFIG_LOGO=y
# CONFIG_LOGO_LINUX_MONO is not set
CONFIG_LOGO_LINUX_VGA16=y
CONFIG_LOGO_LINUX_CLUT224=y
CONFIG_SOUND=m
CONFIG_SOUND_OSS_CORE=y
CONFIG_SOUND_OSS_CORE_PRECLAIM=y
CONFIG_SND=m
CONFIG_SND_TIMER=m
CONFIG_SND_PCM=m
CONFIG_SND_HWDEP=m
CONFIG_SND_RAWMIDI=m
CONFIG_SND_JACK=y
CONFIG_SND_SEQUENCER=m
CONFIG_SND_SEQ_DUMMY=m
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=m
CONFIG_SND_PCM_OSS=m
CONFIG_SND_PCM_OSS_PLUGINS=y
CONFIG_SND_SEQUENCER_OSS=y
CONFIG_SND_HRTIMER=m
CONFIG_SND_SEQ_HRTIMER_DEFAULT=y
CONFIG_SND_DYNAMIC_MINORS=y
CONFIG_SND_SUPPORT_OLD_API=y
CONFIG_SND_VERBOSE_PROCFS=y
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set
CONFIG_SND_VMASTER=y
CONFIG_SND_DMA_SGBUF=y
CONFIG_SND_RAWMIDI_SEQ=m
CONFIG_SND_OPL3_LIB_SEQ=m
CONFIG_SND_OPL4_LIB_SEQ=m
CONFIG_SND_SBAWE_SEQ=m
CONFIG_SND_EMU10K1_SEQ=m
CONFIG_SND_MPU401_UART=m
CONFIG_SND_OPL3_LIB=m
CONFIG_SND_OPL4_LIB=m
CONFIG_SND_VX_LIB=m
CONFIG_SND_AC97_CODEC=m
CONFIG_SND_DRIVERS=y
CONFIG_SND_PCSP=m
CONFIG_SND_DUMMY=m
CONFIG_SND_ALOOP=m
CONFIG_SND_VIRMIDI=m
CONFIG_SND_MTPAV=m
CONFIG_SND_MTS64=m
CONFIG_SND_SERIAL_U16550=m
CONFIG_SND_MPU401=m
CONFIG_SND_PORTMAN2X4=m
CONFIG_SND_AC97_POWER_SAVE=y
CONFIG_SND_AC97_POWER_SAVE_DEFAULT=0
CONFIG_SND_WSS_LIB=m
CONFIG_SND_SB_COMMON=m
CONFIG_SND_SB8_DSP=m
CONFIG_SND_SB16_DSP=m
CONFIG_SND_ISA=y
CONFIG_SND_ADLIB=m
CONFIG_SND_AD1816A=m
CONFIG_SND_AD1848=m
CONFIG_SND_ALS100=m
CONFIG_SND_AZT1605=m
CONFIG_SND_AZT2316=m
CONFIG_SND_AZT2320=m
CONFIG_SND_CMI8330=m
CONFIG_SND_CS4231=m
CONFIG_SND_CS4236=m
CONFIG_SND_ES1688=m
CONFIG_SND_ES18XX=m
CONFIG_SND_SC6000=m
CONFIG_SND_GUSCLASSIC=m
CONFIG_SND_GUSEXTREME=m
CONFIG_SND_GUSMAX=m
CONFIG_SND_INTERWAVE=m
CONFIG_SND_INTERWAVE_STB=m
CONFIG_SND_JAZZ16=m
CONFIG_SND_OPL3SA2=m
CONFIG_SND_OPTI92X_AD1848=m
CONFIG_SND_OPTI92X_CS4231=m
CONFIG_SND_OPTI93X=m
CONFIG_SND_MIRO=m
CONFIG_SND_SB8=m
CONFIG_SND_SB16=m
CONFIG_SND_SBAWE=m
CONFIG_SND_SB16_CSP=y
CONFIG_SND_SSCAPE=m
CONFIG_SND_WAVEFRONT=m
CONFIG_SND_MSND_PINNACLE=m
CONFIG_SND_MSND_CLASSIC=m
CONFIG_SND_PCI=y
CONFIG_SND_AD1889=m
CONFIG_SND_ALS300=m
CONFIG_SND_ALS4000=m
CONFIG_SND_ALI5451=m
CONFIG_SND_ASIHPI=m
CONFIG_SND_ATIIXP=m
CONFIG_SND_ATIIXP_MODEM=m
CONFIG_SND_AU8810=m
CONFIG_SND_AU8820=m
CONFIG_SND_AU8830=m
CONFIG_SND_AW2=m
CONFIG_SND_AZT3328=m
CONFIG_SND_BT87X=m
# CONFIG_SND_BT87X_OVERCLOCK is not set
CONFIG_SND_CA0106=m
CONFIG_SND_CMIPCI=m
CONFIG_SND_OXYGEN_LIB=m
CONFIG_SND_OXYGEN=m
CONFIG_SND_CS4281=m
CONFIG_SND_CS46XX=m
CONFIG_SND_CS46XX_NEW_DSP=y
CONFIG_SND_CS5530=m
CONFIG_SND_CS5535AUDIO=m
CONFIG_SND_CTXFI=m
CONFIG_SND_DARLA20=m
CONFIG_SND_GINA20=m
CONFIG_SND_LAYLA20=m
CONFIG_SND_DARLA24=m
CONFIG_SND_GINA24=m
CONFIG_SND_LAYLA24=m
CONFIG_SND_MONA=m
CONFIG_SND_MIA=m
CONFIG_SND_ECHO3G=m
CONFIG_SND_INDIGO=m
CONFIG_SND_INDIGOIO=m
CONFIG_SND_INDIGODJ=m
CONFIG_SND_INDIGOIOX=m
CONFIG_SND_INDIGODJX=m
CONFIG_SND_EMU10K1=m
CONFIG_SND_EMU10K1X=m
CONFIG_SND_ENS1370=m
CONFIG_SND_ENS1371=m
CONFIG_SND_ES1938=m
CONFIG_SND_ES1968=m
CONFIG_SND_ES1968_INPUT=y
CONFIG_SND_FM801=m
CONFIG_SND_FM801_TEA575X_BOOL=y
CONFIG_SND_FM801_TEA575X=m
CONFIG_SND_HDA_INTEL=m
CONFIG_SND_HDA_HWDEP=y
CONFIG_SND_HDA_RECONFIG=y
CONFIG_SND_HDA_INPUT_BEEP=y
CONFIG_SND_HDA_INPUT_BEEP_MODE=1
CONFIG_SND_HDA_INPUT_JACK=y
# CONFIG_SND_HDA_PATCH_LOADER is not set
CONFIG_SND_HDA_CODEC_REALTEK=y
CONFIG_SND_HDA_CODEC_ANALOG=y
CONFIG_SND_HDA_CODEC_SIGMATEL=y
CONFIG_SND_HDA_CODEC_VIA=y
CONFIG_SND_HDA_CODEC_HDMI=y
CONFIG_SND_HDA_CODEC_CIRRUS=y
CONFIG_SND_HDA_CODEC_CONEXANT=y
CONFIG_SND_HDA_CODEC_CA0110=y
CONFIG_SND_HDA_CODEC_CMEDIA=y
CONFIG_SND_HDA_CODEC_SI3054=y
CONFIG_SND_HDA_GENERIC=y
# CONFIG_SND_HDA_POWER_SAVE is not set
CONFIG_SND_HDSP=m
CONFIG_SND_HDSPM=m
CONFIG_SND_ICE1712=m
CONFIG_SND_ICE1724=m
CONFIG_SND_INTEL8X0=m
CONFIG_SND_INTEL8X0M=m
CONFIG_SND_KORG1212=m
CONFIG_SND_LX6464ES=m
CONFIG_SND_MAESTRO3=m
CONFIG_SND_MAESTRO3_INPUT=y
CONFIG_SND_MIXART=m
CONFIG_SND_NM256=m
CONFIG_SND_PCXHR=m
CONFIG_SND_RIPTIDE=m
CONFIG_SND_RME32=m
CONFIG_SND_RME96=m
CONFIG_SND_RME9652=m
CONFIG_SND_SIS7019=m
CONFIG_SND_SONICVIBES=m
CONFIG_SND_TRIDENT=m
CONFIG_SND_VIA82XX=m
CONFIG_SND_VIA82XX_MODEM=m
CONFIG_SND_VIRTUOSO=m
CONFIG_SND_VX222=m
CONFIG_SND_YMFPCI=m
CONFIG_SND_SPI=y
CONFIG_SND_USB=y
CONFIG_SND_USB_AUDIO=m
CONFIG_SND_USB_UA101=m
CONFIG_SND_USB_USX2Y=m
CONFIG_SND_USB_CAIAQ=m
CONFIG_SND_USB_CAIAQ_INPUT=y
CONFIG_SND_USB_US122L=m
CONFIG_SND_USB_6FIRE=m
CONFIG_SND_FIREWIRE=y
CONFIG_SND_FIREWIRE_LIB=m
CONFIG_SND_FIREWIRE_SPEAKERS=m
CONFIG_SND_PCMCIA=y
CONFIG_SND_VXPOCKET=m
CONFIG_SND_PDAUDIOCF=m
CONFIG_SND_SOC=m
# CONFIG_SND_SOC_CACHE_LZO is not set
CONFIG_SND_SOC_I2C_AND_SPI=m
CONFIG_SND_SOC_ALL_CODECS=m
CONFIG_SND_SOC_88PM860X=m
CONFIG_SND_SOC_WM_HUBS=m
CONFIG_SND_SOC_AD1836=m
CONFIG_SND_SOC_AD193X=m
CONFIG_SND_SOC_AD73311=m
CONFIG_SND_SOC_ADS117X=m
CONFIG_SND_SOC_AK4104=m
CONFIG_SND_SOC_AK4535=m
CONFIG_SND_SOC_AK4642=m
CONFIG_SND_SOC_AK4671=m
CONFIG_SND_SOC_ALC5623=m
CONFIG_SND_SOC_CS42L51=m
CONFIG_SND_SOC_CS4270=m
CONFIG_SND_SOC_CS4271=m
CONFIG_SND_SOC_CX20442=m
CONFIG_SND_SOC_L3=m
CONFIG_SND_SOC_DA7210=m
CONFIG_SND_SOC_DFBMCS320=m
CONFIG_SND_SOC_MAX98088=m
CONFIG_SND_SOC_MAX9850=m
CONFIG_SND_SOC_PCM3008=m
CONFIG_SND_SOC_SGTL5000=m
CONFIG_SND_SOC_SPDIF=m
CONFIG_SND_SOC_SSM2602=m
CONFIG_SND_SOC_TLV320AIC23=m
CONFIG_SND_SOC_TLV320AIC26=m
CONFIG_SND_SOC_TVL320AIC32X4=m
CONFIG_SND_SOC_TLV320AIC3X=m
CONFIG_SND_SOC_TLV320DAC33=m
CONFIG_SND_SOC_TWL4030=m
CONFIG_SND_SOC_TWL6040=m
CONFIG_SND_SOC_UDA134X=m
CONFIG_SND_SOC_UDA1380=m
CONFIG_SND_SOC_WL1273=m
CONFIG_SND_SOC_WM8350=m
CONFIG_SND_SOC_WM8400=m
CONFIG_SND_SOC_WM8510=m
CONFIG_SND_SOC_WM8523=m
CONFIG_SND_SOC_WM8580=m
CONFIG_SND_SOC_WM8711=m
CONFIG_SND_SOC_WM8727=m
CONFIG_SND_SOC_WM8728=m
CONFIG_SND_SOC_WM8731=m
CONFIG_SND_SOC_WM8737=m
CONFIG_SND_SOC_WM8741=m
CONFIG_SND_SOC_WM8750=m
CONFIG_SND_SOC_WM8753=m
CONFIG_SND_SOC_WM8770=m
CONFIG_SND_SOC_WM8776=m
CONFIG_SND_SOC_WM8804=m
CONFIG_SND_SOC_WM8900=m
CONFIG_SND_SOC_WM8903=m
CONFIG_SND_SOC_WM8904=m
CONFIG_SND_SOC_WM8940=m
CONFIG_SND_SOC_WM8955=m
CONFIG_SND_SOC_WM8960=m
CONFIG_SND_SOC_WM8961=m
CONFIG_SND_SOC_WM8962=m
CONFIG_SND_SOC_WM8971=m
CONFIG_SND_SOC_WM8974=m
CONFIG_SND_SOC_WM8978=m
CONFIG_SND_SOC_WM8985=m
CONFIG_SND_SOC_WM8988=m
CONFIG_SND_SOC_WM8990=m
CONFIG_SND_SOC_WM8991=m
CONFIG_SND_SOC_WM8993=m
CONFIG_SND_SOC_WM8994=m
CONFIG_SND_SOC_WM8995=m
CONFIG_SND_SOC_WM9081=m
CONFIG_SND_SOC_LM4857=m
CONFIG_SND_SOC_MAX9877=m
CONFIG_SND_SOC_TPA6130A2=m
CONFIG_SND_SOC_WM2000=m
CONFIG_SND_SOC_WM9090=m
CONFIG_SOUND_PRIME=m
# CONFIG_SOUND_MSNDCLAS is not set
# CONFIG_SOUND_MSNDPIN is not set
CONFIG_SOUND_OSS=m
CONFIG_SOUND_TRACEINIT=y
CONFIG_SOUND_DMAP=y
CONFIG_SOUND_VMIDI=m
CONFIG_SOUND_TRIX=m
CONFIG_SOUND_MSS=m
CONFIG_SOUND_MPU401=m
CONFIG_SOUND_PAS=m
CONFIG_SOUND_PSS=m
CONFIG_PSS_MIXER=y
CONFIG_SOUND_SB=m
CONFIG_SOUND_YM3812=m
CONFIG_SOUND_UART6850=m
CONFIG_SOUND_AEDSP16=m
CONFIG_SC6600=y
CONFIG_SC6600_JOY=y
CONFIG_SC6600_CDROM=4
CONFIG_SC6600_CDROMBASE=0
CONFIG_SOUND_KAHLUA=m
CONFIG_AC97_BUS=m
CONFIG_HID_SUPPORT=y
CONFIG_HID=y
CONFIG_HIDRAW=y

#
# USB Input Devices
#
CONFIG_USB_HID=y
CONFIG_HID_PID=y
CONFIG_USB_HIDDEV=y

#
# Special HID drivers
#
CONFIG_HID_3M_PCT=y
CONFIG_HID_A4TECH=y
CONFIG_HID_ACRUX=m
CONFIG_HID_ACRUX_FF=m
CONFIG_HID_APPLE=y
CONFIG_HID_BELKIN=y
CONFIG_HID_CANDO=y
CONFIG_HID_CHERRY=y
CONFIG_HID_CHICONY=y
CONFIG_HID_PRODIKEYS=m
CONFIG_HID_CYPRESS=y
CONFIG_HID_DRAGONRISE=y
CONFIG_DRAGONRISE_FF=y
CONFIG_HID_EMS_FF=m
CONFIG_HID_ELECOM=m
CONFIG_HID_EZKEY=y
CONFIG_HID_KEYTOUCH=y
CONFIG_HID_KYE=y
CONFIG_HID_UCLOGIC=y
CONFIG_HID_WALTOP=y
CONFIG_HID_GYRATION=y
CONFIG_HID_TWINHAN=y
CONFIG_HID_KENSINGTON=y
CONFIG_HID_LCPOWER=y
CONFIG_HID_LOGITECH=y
CONFIG_LOGITECH_FF=y
CONFIG_LOGIRUMBLEPAD2_FF=y
CONFIG_LOGIG940_FF=y
CONFIG_LOGIWII_FF=y
CONFIG_HID_MAGICMOUSE=m
CONFIG_HID_MICROSOFT=y
CONFIG_HID_MOSART=y
CONFIG_HID_MONTEREY=y
CONFIG_HID_MULTITOUCH=y
CONFIG_HID_NTRIG=y
CONFIG_HID_ORTEK=y
CONFIG_HID_PANTHERLORD=y
CONFIG_PANTHERLORD_FF=y
CONFIG_HID_PETALYNX=y
CONFIG_HID_PICOLCD=m
CONFIG_HID_PICOLCD_FB=y
CONFIG_HID_PICOLCD_BACKLIGHT=y
CONFIG_HID_PICOLCD_LCD=y
CONFIG_HID_PICOLCD_LEDS=y
CONFIG_HID_QUANTA=y
CONFIG_HID_ROCCAT=y
CONFIG_HID_ROCCAT_COMMON=y
CONFIG_HID_ROCCAT_ARVO=y
CONFIG_HID_ROCCAT_KONE=y
CONFIG_HID_ROCCAT_KONEPLUS=y
CONFIG_HID_ROCCAT_KOVAPLUS=y
CONFIG_HID_ROCCAT_PYRA=y
CONFIG_HID_SAMSUNG=y
CONFIG_HID_SONY=y
CONFIG_HID_STANTUM=y
CONFIG_HID_SUNPLUS=y
CONFIG_HID_GREENASIA=y
CONFIG_GREENASIA_FF=y
CONFIG_HID_SMARTJOYPLUS=y
CONFIG_SMARTJOYPLUS_FF=y
CONFIG_HID_TOPSEED=y
CONFIG_HID_THRUSTMASTER=y
CONFIG_THRUSTMASTER_FF=y
CONFIG_HID_WACOM=m
CONFIG_HID_WACOM_POWER_SUPPLY=y
CONFIG_HID_ZEROPLUS=y
CONFIG_ZEROPLUS_FF=y
CONFIG_HID_ZYDACRON=m
CONFIG_USB_SUPPORT=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=y
# CONFIG_USB_DEBUG is not set
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y

#
# Miscellaneous USB options
#
# CONFIG_USB_DEVICEFS is not set
CONFIG_USB_DEVICE_CLASS=y
CONFIG_USB_DYNAMIC_MINORS=y
# CONFIG_USB_SUSPEND is not set
# CONFIG_USB_OTG_WHITELIST is not set
# CONFIG_USB_OTG_BLACKLIST_HUB is not set
CONFIG_USB_MON=y
CONFIG_USB_WUSB=y
CONFIG_USB_WUSB_CBAF=y
# CONFIG_USB_WUSB_CBAF_DEBUG is not set

#
# USB Host Controller Drivers
#
CONFIG_USB_C67X00_HCD=y
CONFIG_USB_XHCI_HCD=y
# CONFIG_USB_XHCI_HCD_DEBUGGING is not set
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_EHCI_ROOT_HUB_TT=y
CONFIG_USB_EHCI_TT_NEWSCHED=y
CONFIG_USB_OXU210HP_HCD=y
CONFIG_USB_ISP116X_HCD=y
CONFIG_USB_ISP1760_HCD=y
CONFIG_USB_ISP1362_HCD=y
CONFIG_USB_OHCI_HCD=y
# CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_UHCI_HCD=y
CONFIG_USB_U132_HCD=m
CONFIG_USB_SL811_HCD=y
CONFIG_USB_SL811_CS=y
CONFIG_USB_R8A66597_HCD=y
CONFIG_USB_WHCI_HCD=y
CONFIG_USB_HWA_HCD=y

#
# USB Device Class drivers
#
CONFIG_USB_ACM=y
CONFIG_USB_PRINTER=y
CONFIG_USB_WDM=y
CONFIG_USB_TMC=y

#
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may #

# also be needed; see USB_STORAGE Help for more info
#
CONFIG_USB_STORAGE=y
# CONFIG_USB_STORAGE_DEBUG is not set
CONFIG_USB_STORAGE_REALTEK=y
CONFIG_USB_STORAGE_DATAFAB=y
CONFIG_USB_STORAGE_FREECOM=y
CONFIG_USB_STORAGE_ISD200=y
CONFIG_USB_STORAGE_USBAT=y
CONFIG_USB_STORAGE_SDDR09=y
CONFIG_USB_STORAGE_SDDR55=y
CONFIG_USB_STORAGE_JUMPSHOT=y
CONFIG_USB_STORAGE_ALAUDA=y
CONFIG_USB_STORAGE_ONETOUCH=y
CONFIG_USB_STORAGE_KARMA=y
CONFIG_USB_STORAGE_CYPRESS_ATACB=y
CONFIG_USB_STORAGE_ENE_UB6250=y
CONFIG_USB_UAS=y
CONFIG_USB_LIBUSUAL=y

#
# USB Imaging devices
#
CONFIG_USB_MDC800=m
CONFIG_USB_MICROTEK=m

#
# USB port drivers
#
CONFIG_USB_USS720=m
CONFIG_USB_SERIAL=m
CONFIG_USB_EZUSB=y
CONFIG_USB_SERIAL_GENERIC=y
CONFIG_USB_SERIAL_AIRCABLE=m
CONFIG_USB_SERIAL_ARK3116=m
CONFIG_USB_SERIAL_BELKIN=m
CONFIG_USB_SERIAL_CH341=m
CONFIG_USB_SERIAL_WHITEHEAT=m
CONFIG_USB_SERIAL_DIGI_ACCELEPORT=m
CONFIG_USB_SERIAL_CP210X=m
CONFIG_USB_SERIAL_CYPRESS_M8=m
CONFIG_USB_SERIAL_EMPEG=m
CONFIG_USB_SERIAL_FTDI_SIO=m
CONFIG_USB_SERIAL_FUNSOFT=m
CONFIG_USB_SERIAL_VISOR=m
CONFIG_USB_SERIAL_IPAQ=m
CONFIG_USB_SERIAL_IR=m
CONFIG_USB_SERIAL_EDGEPORT=m
CONFIG_USB_SERIAL_EDGEPORT_TI=m
CONFIG_USB_SERIAL_GARMIN=m
CONFIG_USB_SERIAL_IPW=m
CONFIG_USB_SERIAL_IUU=m
CONFIG_USB_SERIAL_KEYSPAN_PDA=m
CONFIG_USB_SERIAL_KEYSPAN=m
CONFIG_USB_SERIAL_KLSI=m
CONFIG_USB_SERIAL_KOBIL_SCT=m
CONFIG_USB_SERIAL_MCT_U232=m
CONFIG_USB_SERIAL_MOS7720=m
CONFIG_USB_SERIAL_MOS7715_PARPORT=y
CONFIG_USB_SERIAL_MOS7840=m
CONFIG_USB_SERIAL_MOTOROLA=m
CONFIG_USB_SERIAL_NAVMAN=m
CONFIG_USB_SERIAL_PL2303=m
CONFIG_USB_SERIAL_OTI6858=m
CONFIG_USB_SERIAL_QCAUX=m
CONFIG_USB_SERIAL_QUALCOMM=m
CONFIG_USB_SERIAL_SPCP8X5=m
CONFIG_USB_SERIAL_HP4X=m
CONFIG_USB_SERIAL_SAFE=m
CONFIG_USB_SERIAL_SAFE_PADDED=y
CONFIG_USB_SERIAL_SAMBA=m
CONFIG_USB_SERIAL_SIEMENS_MPI=m
CONFIG_USB_SERIAL_SIERRAWIRELESS=m
CONFIG_USB_SERIAL_SYMBOL=m
CONFIG_USB_SERIAL_TI=m
CONFIG_USB_SERIAL_CYBERJACK=m
CONFIG_USB_SERIAL_XIRCOM=m
CONFIG_USB_SERIAL_WWAN=m
CONFIG_USB_SERIAL_OPTION=m
CONFIG_USB_SERIAL_OMNINET=m
CONFIG_USB_SERIAL_OPTICON=m
CONFIG_USB_SERIAL_VIVOPAY_SERIAL=m
CONFIG_USB_SERIAL_ZIO=m
CONFIG_USB_SERIAL_SSU100=m
CONFIG_USB_SERIAL_DEBUG=m

#
# USB Miscellaneous drivers
#
CONFIG_USB_EMI62=m
CONFIG_USB_EMI26=m
CONFIG_USB_ADUTUX=m
CONFIG_USB_SEVSEG=m
CONFIG_USB_RIO500=m
CONFIG_USB_LEGOTOWER=m
CONFIG_USB_LCD=m
CONFIG_USB_LED=m
CONFIG_USB_CYPRESS_CY7C63=m
CONFIG_USB_CYTHERM=m
CONFIG_USB_IDMOUSE=m
CONFIG_USB_FTDI_ELAN=m
CONFIG_USB_APPLEDISPLAY=m
CONFIG_USB_SISUSBVGA=m
CONFIG_USB_SISUSBVGA_CON=y
CONFIG_USB_LD=m
CONFIG_USB_TRANCEVIBRATOR=m
CONFIG_USB_IOWARRIOR=m
# CONFIG_USB_TEST is not set
CONFIG_USB_ISIGHTFW=m
CONFIG_USB_YUREX=m
CONFIG_USB_ATM=m
CONFIG_USB_SPEEDTOUCH=m
CONFIG_USB_CXACRU=m
CONFIG_USB_UEAGLEATM=m
CONFIG_USB_XUSBATM=m
CONFIG_USB_GADGET=m
# CONFIG_USB_GADGET_DEBUG_FILES is not set
# CONFIG_USB_GADGET_DEBUG_FS is not set
CONFIG_USB_GADGET_VBUS_DRAW=2
CONFIG_USB_GADGET_SELECTED=y
# CONFIG_USB_GADGET_FUSB300 is not set
# CONFIG_USB_GADGET_R8A66597 is not set
# CONFIG_USB_GADGET_PXA_U2O is not set
# CONFIG_USB_GADGET_M66592 is not set
# CONFIG_USB_GADGET_AMD5536UDC is not set
CONFIG_USB_GADGET_CI13XXX_PCI=y
CONFIG_USB_CI13XXX_PCI=m
# CONFIG_USB_GADGET_NET2280 is not set
# CONFIG_USB_GADGET_GOKU is not set
# CONFIG_USB_GADGET_LANGWELL is not set
# CONFIG_USB_GADGET_EG20T is not set
# CONFIG_USB_GADGET_DUMMY_HCD is not set
CONFIG_USB_GADGET_DUALSPEED=y
CONFIG_USB_ZERO=m
CONFIG_USB_AUDIO=m
CONFIG_USB_ETH=m
CONFIG_USB_ETH_RNDIS=y
CONFIG_USB_ETH_EEM=y
CONFIG_USB_G_NCM=m
CONFIG_USB_GADGETFS=m
CONFIG_USB_FUNCTIONFS=m
CONFIG_USB_FUNCTIONFS_ETH=y
CONFIG_USB_FUNCTIONFS_RNDIS=y
CONFIG_USB_FUNCTIONFS_GENERIC=y
CONFIG_USB_FILE_STORAGE=m
CONFIG_USB_FILE_STORAGE_TEST=y
CONFIG_USB_MASS_STORAGE=m
CONFIG_USB_G_SERIAL=m
CONFIG_USB_MIDI_GADGET=m
CONFIG_USB_G_PRINTER=m
CONFIG_USB_CDC_COMPOSITE=m
CONFIG_USB_G_NOKIA=m
CONFIG_USB_G_MULTI=m
CONFIG_USB_G_MULTI_RNDIS=y
CONFIG_USB_G_MULTI_CDC=y
CONFIG_USB_G_HID=m
CONFIG_USB_G_DBGP=m
CONFIG_USB_G_DBGP_PRINTK=y
# CONFIG_USB_G_DBGP_SERIAL is not set
CONFIG_USB_G_WEBCAM=m

#
# OTG and related infrastructure
#
CONFIG_USB_OTG_UTILS=y
CONFIG_USB_GPIO_VBUS=m
CONFIG_TWL4030_USB=m
CONFIG_TWL6030_USB=m
CONFIG_NOP_USB_XCEIV=m
CONFIG_AB8500_USB=m
CONFIG_UWB=y
CONFIG_UWB_HWA=y
CONFIG_UWB_WHCI=y
CONFIG_UWB_I1480U=m
CONFIG_MMC=y
# CONFIG_MMC_DEBUG is not set
# CONFIG_MMC_UNSAFE_RESUME is not set
# CONFIG_MMC_CLKGATE is not set

#
# MMC/SD/SDIO Card Drivers
#
CONFIG_MMC_BLOCK=y
CONFIG_MMC_BLOCK_MINORS=8
CONFIG_MMC_BLOCK_BOUNCE=y
CONFIG_SDIO_UART=y
CONFIG_MMC_TEST=y

#
# MMC/SD/SDIO Host Controller Drivers
#
CONFIG_MMC_SDHCI=y
CONFIG_MMC_SDHCI_PCI=y
CONFIG_MMC_RICOH_MMC=y
CONFIG_MMC_SDHCI_OF=y
CONFIG_MMC_SDHCI_PLTFM=y
CONFIG_MMC_WBSD=y
CONFIG_MMC_TIFM_SD=y
CONFIG_MMC_SDRICOH_CS=y
CONFIG_MMC_CB710=y
CONFIG_MMC_VIA_SDMMC=y
CONFIG_MMC_USHC=y
CONFIG_MEMSTICK=y
# CONFIG_MEMSTICK_DEBUG is not set

#
# MemoryStick drivers
#
# CONFIG_MEMSTICK_UNSAFE_RESUME is not set
CONFIG_MSPRO_BLOCK=y

#
# MemoryStick Host Controller Drivers
#
CONFIG_MEMSTICK_TIFM_MS=y
CONFIG_MEMSTICK_JMICRON_38X=y
CONFIG_MEMSTICK_R592=y
CONFIG_NEW_LEDS=y
CONFIG_LEDS_CLASS=y

#
# LED drivers
#
CONFIG_LEDS_88PM860X=m
CONFIG_LEDS_LM3530=m
CONFIG_LEDS_NET48XX=m
CONFIG_LEDS_NET5501=m
CONFIG_LEDS_WRAP=m
CONFIG_LEDS_ALIX2=m
CONFIG_LEDS_PCA9532=m
CONFIG_LEDS_GPIO=m
CONFIG_LEDS_GPIO_PLATFORM=y
CONFIG_LEDS_GPIO_OF=y
CONFIG_LEDS_LP3944=m
CONFIG_LEDS_LP5521=m
CONFIG_LEDS_LP5523=m
CONFIG_LEDS_CLEVO_MAIL=m
CONFIG_LEDS_PCA955X=m
CONFIG_LEDS_WM831X_STATUS=m
CONFIG_LEDS_WM8350=m
CONFIG_LEDS_DA903X=m
CONFIG_LEDS_DAC124S085=m
CONFIG_LEDS_REGULATOR=m
CONFIG_LEDS_BD2802=m
CONFIG_LEDS_INTEL_SS4200=m
CONFIG_LEDS_LT3593=m
CONFIG_LEDS_ADP5520=m
CONFIG_LEDS_DELL_NETBOOKS=m
CONFIG_LEDS_MC13783=m
CONFIG_LEDS_TRIGGERS=y

#
# LED Triggers
#
CONFIG_LEDS_TRIGGER_TIMER=m
CONFIG_LEDS_TRIGGER_HEARTBEAT=m
CONFIG_LEDS_TRIGGER_BACKLIGHT=m
CONFIG_LEDS_TRIGGER_GPIO=m
CONFIG_LEDS_TRIGGER_DEFAULT_ON=m

# iptables trigger is under Netfilter config (LED target)
#
CONFIG_NFC_DEVICES=y
CONFIG_PN544_NFC=m
CONFIG_ACCESSIBILITY=y
CONFIG_A11Y_BRAILLE_CONSOLE=y
CONFIG_INFINIBAND=m
CONFIG_INFINIBAND_USER_MAD=m
CONFIG_INFINIBAND_USER_ACCESS=m
CONFIG_INFINIBAND_USER_MEM=y
CONFIG_INFINIBAND_ADDR_TRANS=y
CONFIG_INFINIBAND_MTHCA=m
CONFIG_INFINIBAND_MTHCA_DEBUG=y
CONFIG_INFINIBAND_AMSO1100=m
# CONFIG_INFINIBAND_AMSO1100_DEBUG is not set
CONFIG_INFINIBAND_CXGB3=m
# CONFIG_INFINIBAND_CXGB3_DEBUG is not set
CONFIG_INFINIBAND_CXGB4=m
CONFIG_MLX4_INFINIBAND=m
CONFIG_INFINIBAND_NES=m
# CONFIG_INFINIBAND_NES_DEBUG is not set
CONFIG_INFINIBAND_IPOIB=m
CONFIG_INFINIBAND_IPOIB_CM=y
# CONFIG_INFINIBAND_IPOIB_DEBUG is not set
CONFIG_INFINIBAND_SRP=m
CONFIG_INFINIBAND_ISER=m
CONFIG_EDAC=y

#
# Reporting subsystems
#
# CONFIG_EDAC_DEBUG is not set
CONFIG_EDAC_DECODE_MCE=y
CONFIG_EDAC_MCE_INJ=m
CONFIG_EDAC_MM_EDAC=y
CONFIG_EDAC_MCE=y
CONFIG_EDAC_AMD76X=m
CONFIG_EDAC_E7XXX=m
CONFIG_EDAC_E752X=m
CONFIG_EDAC_I82875P=m
CONFIG_EDAC_I82975X=m
CONFIG_EDAC_I3000=m
CONFIG_EDAC_I3200=m
CONFIG_EDAC_X38=m
CONFIG_EDAC_I5400=m
CONFIG_EDAC_I7CORE=m
CONFIG_EDAC_I82860=m
CONFIG_EDAC_R82600=m
CONFIG_EDAC_I5000=m
CONFIG_EDAC_I5100=m
CONFIG_EDAC_I7300=m
CONFIG_RTC_LIB=m
CONFIG_RTC_CLASS=m

#
# RTC interfaces
#
CONFIG_RTC_INTF_SYSFS=y
CONFIG_RTC_INTF_PROC=y
CONFIG_RTC_INTF_DEV=y
# CONFIG_RTC_INTF_DEV_UIE_EMUL is not set
CONFIG_RTC_DRV_TEST=m

#
# I2C RTC drivers
#
CONFIG_RTC_DRV_DS1307=m
CONFIG_RTC_DRV_DS1374=m
CONFIG_RTC_DRV_DS1672=m
CONFIG_RTC_DRV_DS3232=m
CONFIG_RTC_DRV_MAX6900=m
CONFIG_RTC_DRV_MAX8925=m
CONFIG_RTC_DRV_MAX8998=m
CONFIG_RTC_DRV_RS5C372=m
CONFIG_RTC_DRV_ISL1208=m
CONFIG_RTC_DRV_ISL12022=m
CONFIG_RTC_DRV_X1205=m
CONFIG_RTC_DRV_PCF8563=m
CONFIG_RTC_DRV_PCF8583=m
CONFIG_RTC_DRV_M41T80=m
CONFIG_RTC_DRV_M41T80_WDT=y
CONFIG_RTC_DRV_BQ32K=m
CONFIG_RTC_DRV_TWL4030=m
CONFIG_RTC_DRV_S35390A=m
CONFIG_RTC_DRV_FM3130=m
CONFIG_RTC_DRV_RX8581=m
CONFIG_RTC_DRV_RX8025=m

#
# SPI RTC drivers
#
CONFIG_RTC_DRV_M41T94=m
CONFIG_RTC_DRV_DS1305=m
CONFIG_RTC_DRV_DS1390=m
CONFIG_RTC_DRV_MAX6902=m
CONFIG_RTC_DRV_R9701=m
CONFIG_RTC_DRV_RS5C348=m
CONFIG_RTC_DRV_DS3234=m
CONFIG_RTC_DRV_PCF2123=m

#
# Platform RTC drivers
#
CONFIG_RTC_DRV_CMOS=m
CONFIG_RTC_DRV_DS1286=m
CONFIG_RTC_DRV_DS1511=m
CONFIG_RTC_DRV_DS1553=m
CONFIG_RTC_DRV_DS1742=m
CONFIG_RTC_DRV_STK17TA8=m
CONFIG_RTC_DRV_M48T86=m
CONFIG_RTC_DRV_M48T35=m
CONFIG_RTC_DRV_M48T59=m
CONFIG_RTC_DRV_MSM6242=m
CONFIG_RTC_DRV_BQ4802=m
CONFIG_RTC_DRV_RP5C01=m
CONFIG_RTC_DRV_V3020=m
CONFIG_RTC_DRV_WM831X=m
CONFIG_RTC_DRV_WM8350=m
CONFIG_RTC_DRV_PCF50633=m
CONFIG_RTC_DRV_AB3100=m
CONFIG_RTC_DRV_AB8500=m

#
# on-CPU RTC drivers
#
CONFIG_RTC_DRV_PCAP=m
CONFIG_RTC_DRV_MC13XXX=m
CONFIG_DMADEVICES=y
# CONFIG_DMADEVICES_DEBUG is not set

#
# DMA Devices
#
CONFIG_INTEL_MID_DMAC=y
CONFIG_INTEL_IOATDMA=y
CONFIG_TIMB_DMA=y
CONFIG_PCH_DMA=y
CONFIG_DMA_ENGINE=y

#
# DMA Clients
#
CONFIG_NET_DMA=y
CONFIG_ASYNC_TX_DMA=y
# CONFIG_DMATEST is not set
CONFIG_DCA=y
CONFIG_AUXDISPLAY=y
CONFIG_KS0108=m
CONFIG_KS0108_PORT=0x378
CONFIG_KS0108_DELAY=2
CONFIG_CFAG12864B=m
CONFIG_CFAG12864B_RATE=20
CONFIG_UIO=y
CONFIG_UIO_CIF=m
CONFIG_UIO_PDRV=m
CONFIG_UIO_PDRV_GENIRQ=m
CONFIG_UIO_AEC=m
CONFIG_UIO_SERCOS3=m
CONFIG_UIO_PCI_GENERIC=m
CONFIG_UIO_NETX=m
CONFIG_STAGING=y
# CONFIG_STAGING_EXCLUDE_BUILD is not set
CONFIG_STALLION=m
CONFIG_ISTALLION=m
CONFIG_DIGIEPCA=m
CONFIG_RISCOM8=m
CONFIG_SPECIALIX=m
CONFIG_COMPUTONE=m
CONFIG_ET131X=m
# CONFIG_ET131X_DEBUG is not set
CONFIG_SLICOSS=m
CONFIG_VIDEO_GO7007=m
CONFIG_VIDEO_GO7007_USB=m
CONFIG_VIDEO_GO7007_USB_S2250_BOARD=m
CONFIG_VIDEO_GO7007_OV7640=m
CONFIG_VIDEO_GO7007_SAA7113=m
CONFIG_VIDEO_GO7007_SAA7115=m
CONFIG_VIDEO_GO7007_TW9903=m
CONFIG_VIDEO_GO7007_UDA1342=m
CONFIG_VIDEO_GO7007_SONY_TUNER=m
CONFIG_VIDEO_GO7007_TW2804=m
CONFIG_VIDEO_CX25821=m
CONFIG_VIDEO_CX25821_ALSA=m
CONFIG_VIDEO_TM6000=m
CONFIG_VIDEO_TM6000_ALSA=m
CONFIG_VIDEO_TM6000_DVB=m
CONFIG_DVB_CXD2099=m
CONFIG_USB_IP_COMMON=m
CONFIG_USB_IP_VHCI_HCD=m
CONFIG_USB_IP_HOST=m
# CONFIG_USB_IP_DEBUG_ENABLE is not set
CONFIG_W35UND=m
CONFIG_PRISM2_USB=m
CONFIG_ECHO=m
CONFIG_BRCM80211=m
CONFIG_BRCMSMAC=y
CONFIG_BRCMFMAC=y
CONFIG_BRCMDBG=y
CONFIG_RT2860=m
CONFIG_RT2870=m
CONFIG_COMEDI=m
# CONFIG_COMEDI_DEBUG is not set
CONFIG_COMEDI_MISC_DRIVERS=m
CONFIG_COMEDI_KCOMEDILIB=m
CONFIG_COMEDI_BOND=m
CONFIG_COMEDI_TEST=m
CONFIG_COMEDI_PARPORT=m
CONFIG_COMEDI_SERIAL2002=m
CONFIG_COMEDI_SKEL=m
CONFIG_COMEDI_ISA_DRIVERS=m
CONFIG_COMEDI_ACL7225B=m
CONFIG_COMEDI_PCL711=m
CONFIG_COMEDI_PCL724=m
CONFIG_COMEDI_PCL725=m
CONFIG_COMEDI_PCL726=m
CONFIG_COMEDI_PCL730=m
CONFIG_COMEDI_PCL812=m
CONFIG_COMEDI_PCL816=m
CONFIG_COMEDI_PCL818=m
CONFIG_COMEDI_PCM3724=m
CONFIG_COMEDI_PCM3730=m
CONFIG_COMEDI_RTI800=m
CONFIG_COMEDI_RTI802=m
CONFIG_COMEDI_DAS16M1=m
CONFIG_COMEDI_DAS16=m
CONFIG_COMEDI_DAS800=m
CONFIG_COMEDI_DAS1800=m
CONFIG_COMEDI_DAS6402=m
CONFIG_COMEDI_DT2801=m
CONFIG_COMEDI_DT2811=m
CONFIG_COMEDI_DT2814=m
CONFIG_COMEDI_DT2815=m
CONFIG_COMEDI_DT2817=m
CONFIG_COMEDI_DT282X=m
CONFIG_COMEDI_DMM32AT=m
CONFIG_COMEDI_FL512=m
CONFIG_COMEDI_AIO_AIO12_8=m
CONFIG_COMEDI_AIO_IIRO_16=m
CONFIG_COMEDI_C6XDIGIO=m
CONFIG_COMEDI_MPC624=m
CONFIG_COMEDI_ADQ12B=m
CONFIG_COMEDI_NI_AT_A2150=m
CONFIG_COMEDI_NI_AT_AO=m
CONFIG_COMEDI_NI_ATMIO=m
CONFIG_COMEDI_NI_ATMIO16D=m
CONFIG_COMEDI_PCMAD=m
CONFIG_COMEDI_PCMDA12=m
CONFIG_COMEDI_PCMMIO=m
CONFIG_COMEDI_PCMUIO=m
CONFIG_COMEDI_MULTIQ3=m
CONFIG_COMEDI_POC=m
CONFIG_COMEDI_PCI_DRIVERS=m
CONFIG_COMEDI_ADDI_APCI_035=m
CONFIG_COMEDI_ADDI_APCI_1032=m
CONFIG_COMEDI_ADDI_APCI_1500=m
CONFIG_COMEDI_ADDI_APCI_1516=m
CONFIG_COMEDI_ADDI_APCI_1564=m
CONFIG_COMEDI_ADDI_APCI_16XX=m
CONFIG_COMEDI_ADDI_APCI_2016=m
CONFIG_COMEDI_ADDI_APCI_2032=m
CONFIG_COMEDI_ADDI_APCI_2200=m
CONFIG_COMEDI_ADDI_APCI_3001=m
CONFIG_COMEDI_ADDI_APCI_3120=m
CONFIG_COMEDI_ADDI_APCI_3501=m
CONFIG_COMEDI_ADDI_APCI_3XXX=m
CONFIG_COMEDI_ADL_PCI6208=m
CONFIG_COMEDI_ADL_PCI7230=m
CONFIG_COMEDI_ADL_PCI7296=m
CONFIG_COMEDI_ADL_PCI7432=m
CONFIG_COMEDI_ADL_PCI8164=m
CONFIG_COMEDI_ADL_PCI9111=m
CONFIG_COMEDI_ADL_PCI9118=m
CONFIG_COMEDI_ADV_PCI1710=m
CONFIG_COMEDI_ADV_PCI1723=m
CONFIG_COMEDI_ADV_PCI_DIO=m
CONFIG_COMEDI_AMPLC_DIO200=m
CONFIG_COMEDI_AMPLC_PC236=m
CONFIG_COMEDI_AMPLC_PC263=m
CONFIG_COMEDI_AMPLC_PCI224=m
CONFIG_COMEDI_AMPLC_PCI230=m
CONFIG_COMEDI_CONTEC_PCI_DIO=m
CONFIG_COMEDI_DT3000=m
CONFIG_COMEDI_UNIOXX5=m
CONFIG_COMEDI_GSC_HPDI=m
CONFIG_COMEDI_ICP_MULTI=m
CONFIG_COMEDI_II_PCI20KC=m
CONFIG_COMEDI_DAQBOARD2000=m
CONFIG_COMEDI_JR3_PCI=m
CONFIG_COMEDI_KE_COUNTER=m
CONFIG_COMEDI_CB_PCIDAS64=m
CONFIG_COMEDI_CB_PCIDAS=m
CONFIG_COMEDI_CB_PCIDDA=m
CONFIG_COMEDI_CB_PCIDIO=m
CONFIG_COMEDI_CB_PCIMDAS=m
CONFIG_COMEDI_CB_PCIMDDA=m
CONFIG_COMEDI_ME4000=m
CONFIG_COMEDI_ME_DAQ=m
CONFIG_COMEDI_NI_6527=m
CONFIG_COMEDI_NI_65XX=m
CONFIG_COMEDI_NI_660X=m
CONFIG_COMEDI_NI_670X=m
CONFIG_COMEDI_NI_PCIDIO=m
CONFIG_COMEDI_NI_PCIMIO=m
CONFIG_COMEDI_RTD520=m
CONFIG_COMEDI_S526=m
CONFIG_COMEDI_S626=m
CONFIG_COMEDI_SSV_DNP=m
CONFIG_COMEDI_PCMCIA_DRIVERS=m
CONFIG_COMEDI_CB_DAS16_CS=m
CONFIG_COMEDI_DAS08_CS=m
CONFIG_COMEDI_NI_DAQ_700_CS=m
CONFIG_COMEDI_NI_DAQ_DIO24_CS=m
CONFIG_COMEDI_NI_LABPC_CS=m
CONFIG_COMEDI_NI_MIO_CS=m
CONFIG_COMEDI_QUATECH_DAQP_CS=m
CONFIG_COMEDI_USB_DRIVERS=m
CONFIG_COMEDI_DT9812=m
CONFIG_COMEDI_USBDUX=m
CONFIG_COMEDI_USBDUXFAST=m
CONFIG_COMEDI_VMK80XX=m
CONFIG_COMEDI_NI_COMMON=m
CONFIG_COMEDI_MITE=m
CONFIG_COMEDI_NI_TIO=m
CONFIG_COMEDI_NI_LABPC=m
CONFIG_COMEDI_8255=m
CONFIG_COMEDI_DAS08=m
CONFIG_COMEDI_FC=m
CONFIG_FB_OLPC_DCON=m
CONFIG_FB_OLPC_DCON_1=y
CONFIG_FB_OLPC_DCON_1_5=y
CONFIG_ASUS_OLED=m
CONFIG_PANEL=m
CONFIG_PANEL_PARPORT=0
CONFIG_PANEL_PROFILE=5
# CONFIG_PANEL_CHANGE_MESSAGE is not set
CONFIG_R8187SE=m
CONFIG_RTL8192U=m
CONFIG_RTL8192E=m
CONFIG_R8712U=m
CONFIG_R8712_AP=y
CONFIG_RTS_PSTOR=m
# CONFIG_RTS_PSTOR_DEBUG is not set
CONFIG_TRANZPORT=m
CONFIG_POHMELFS=m
# CONFIG_POHMELFS_DEBUG is not set
CONFIG_POHMELFS_CRYPTO=y
CONFIG_IDE_PHISON=m
CONFIG_LINE6_USB=m
CONFIG_LINE6_USB_DEBUG=y
CONFIG_LINE6_USB_DUMP_CTRL=y
CONFIG_LINE6_USB_DUMP_MIDI=y
CONFIG_LINE6_USB_DUMP_PCM=y
CONFIG_LINE6_USB_RAW=y
CONFIG_LINE6_USB_IMPULSE_RESPONSE=y
CONFIG_DRM_VMWGFX=m
CONFIG_DRM_NOUVEAU=m
CONFIG_DRM_NOUVEAU_BACKLIGHT=y
CONFIG_DRM_NOUVEAU_DEBUG=y

#
# I2C encoder or helper chips
#
CONFIG_DRM_I2C_CH7006=m
CONFIG_DRM_I2C_SIL164=m
CONFIG_USB_SERIAL_QUATECH2=m
CONFIG_USB_SERIAL_QUATECH_USB2=m
CONFIG_VT6655=m
CONFIG_VT6656=m
# CONFIG_HYPERV is not set
CONFIG_VME_BUS=m

#
# VME Bridge Drivers
#
CONFIG_VME_CA91CX42=m
CONFIG_VME_TSI148=m

#
# VME Device Drivers
#
CONFIG_VME_USER=m

#
# VME Board Drivers
#
CONFIG_VMIVME_7805=m
CONFIG_DX_SEP=m
CONFIG_IIO=m
CONFIG_IIO_RING_BUFFER=y
CONFIG_IIO_SW_RING=m
CONFIG_IIO_KFIFO_BUF=m
CONFIG_IIO_TRIGGER=y

#
# Accelerometers
#
CONFIG_ADIS16201=m
CONFIG_ADIS16203=m
CONFIG_ADIS16204=m
CONFIG_ADIS16209=m
CONFIG_ADIS16220=m
CONFIG_ADIS16240=m
CONFIG_KXSD9=m
CONFIG_LIS3L02DQ=m
CONFIG_LIS3L02DQ_BUF_KFIFO=y
# CONFIG_LIS3L02DQ_BUF_RING_SW is not set
CONFIG_SCA3000=m

#
# Analog to digital convertors
#
CONFIG_MAX1363=m
CONFIG_MAX1363_RING_BUFFER=y
CONFIG_AD7150=m
CONFIG_AD7152=m
CONFIG_AD7291=m
CONFIG_AD7298=m
CONFIG_AD7314=m
CONFIG_AD7606=m
CONFIG_AD7606_IFACE_PARALLEL=m
CONFIG_AD7606_IFACE_SPI=m
CONFIG_AD799X=m
CONFIG_AD799X_RING_BUFFER=y
CONFIG_AD7476=m
CONFIG_AD7887=m
CONFIG_AD7745=m
CONFIG_AD7816=m
CONFIG_ADT75=m
CONFIG_ADT7310=m
CONFIG_ADT7410=m

#
# Analog digital bi-direction convertors
#
CONFIG_ADT7316=m
CONFIG_ADT7316_SPI=m
CONFIG_ADT7316_I2C=m

#
# Digital to analog convertors
#
CONFIG_AD5624R_SPI=m
CONFIG_AD5446=m
CONFIG_MAX517=m

#
# Direct Digital Synthesis
#
CONFIG_AD5930=m
CONFIG_AD9832=m
CONFIG_AD9834=m
CONFIG_AD9850=m
CONFIG_AD9852=m
CONFIG_AD9910=m
CONFIG_AD9951=m

#
# Digital gyroscope sensors
#
CONFIG_ADIS16060=m
CONFIG_ADIS16080=m
CONFIG_ADIS16130=m
CONFIG_ADIS16260=m

#
# Inertial measurement units
#
CONFIG_ADIS16300=m
CONFIG_ADIS16350=m
CONFIG_ADIS16400=m

#
# Light sensors
#
CONFIG_SENSORS_TSL2563=m
CONFIG_SENSORS_ISL29018=m

#
# Magnetometer sensors
#
CONFIG_SENSORS_AK8975=m
CONFIG_SENSORS_HMC5843=m

#
# Active energy metering IC
#
CONFIG_ADE7753=m
CONFIG_ADE7754=m
CONFIG_ADE7758=m
CONFIG_ADE7759=m
CONFIG_ADE7854=m
CONFIG_ADE7854_I2C=m
CONFIG_ADE7854_SPI=m

#
# Resolver to digital converters
#
CONFIG_AD2S90=m
CONFIG_AD2S120X=m
CONFIG_AD2S1210=m
# CONFIG_AD2S1210_GPIO_INPUT is not set
# CONFIG_AD2S1210_GPIO_OUTPUT is not set
CONFIG_AD2S1210_GPIO_NONE=y

#
# Triggers - standalone
#
CONFIG_IIO_PERIODIC_RTC_TRIGGER=m
CONFIG_IIO_GPIO_TRIGGER=m
CONFIG_IIO_SYSFS_TRIGGER=m
CONFIG_CS5535_GPIO=m
CONFIG_XVMALLOC=y
CONFIG_ZRAM=m
# CONFIG_ZRAM_DEBUG is not set
CONFIG_WLAGS49_H2=m
CONFIG_WLAGS49_H25=m
CONFIG_FB_SM7XX=y
CONFIG_VIDEO_DT3155=m
CONFIG_DT3155_CCIR=y
CONFIG_CRYSTALHD=m
CONFIG_CXT1E1=m
CONFIG_SBE_PMCC4_NCOMM=y
CONFIG_FB_XGI=y
CONFIG_LIRC_STAGING=y
CONFIG_LIRC_BT829=m
CONFIG_LIRC_IGORPLUGUSB=m
CONFIG_LIRC_IMON=m
CONFIG_LIRC_PARALLEL=m
CONFIG_LIRC_SASEM=m
CONFIG_LIRC_SERIAL=m
CONFIG_LIRC_SERIAL_TRANSMITTER=y
CONFIG_LIRC_SIR=m
CONFIG_LIRC_TTUSBIR=m
CONFIG_LIRC_ZILOG=m
CONFIG_EASYCAP=m
CONFIG_EASYCAP_SND=y
# CONFIG_EASYCAP_OSS is not set
# CONFIG_EASYCAP_DEBUG is not set
CONFIG_SOLO6X10=m
CONFIG_ACPI_QUICKSTART=y
CONFIG_MACH_NO_WESTBRIDGE=y
CONFIG_SBE_2T3E3=m
CONFIG_ATH6K_LEGACY=m
CONFIG_AR600x_SD31_XXX=y
# CONFIG_AR600x_WB31_XXX is not set
# CONFIG_AR600x_SD32_XXX is not set
# CONFIG_AR600x_CUSTOM_XXX is not set
CONFIG_ATH6KL_ENABLE_COEXISTENCE=y
CONFIG_AR600x_DUAL_ANTENNA=y
# CONFIG_AR600x_SINGLE_ANTENNA is not set
# CONFIG_AR600x_BT_QCOM is not set
# CONFIG_AR600x_BT_CSR is not set
CONFIG_AR600x_BT_AR3001=y
CONFIG_ATH6KL_HCI_BRIDGE=y
CONFIG_ATH6KL_CONFIG_GPIO_BT_RESET=y
CONFIG_AR600x_BT_RESET_PIN=22
CONFIG_ATH6KL_CFG80211=y
CONFIG_ATH6KL_HTC_RAW_INTERFACE=y
CONFIG_ATH6KL_VIRTUAL_SCATTER_GATHER=y
CONFIG_ATH6KL_SKIP_ABI_VERSION_CHECK=y
# CONFIG_ATH6KL_DEBUG is not set
CONFIG_USB_ENESTORAGE=m
CONFIG_BCM_WIMAX=m
CONFIG_FT1000=m
CONFIG_FT1000_USB=m
CONFIG_FT1000_PCMCIA=m

#
# Speakup console speech
#
CONFIG_SPEAKUP=m
CONFIG_SPEAKUP_SYNTH_ACNTSA=m
CONFIG_SPEAKUP_SYNTH_ACNTPC=m
CONFIG_SPEAKUP_SYNTH_APOLLO=m
CONFIG_SPEAKUP_SYNTH_AUDPTR=m
CONFIG_SPEAKUP_SYNTH_BNS=m
CONFIG_SPEAKUP_SYNTH_DECTLK=m
CONFIG_SPEAKUP_SYNTH_DECEXT=m
CONFIG_SPEAKUP_SYNTH_DECPC=m
CONFIG_SPEAKUP_SYNTH_DTLK=m
CONFIG_SPEAKUP_SYNTH_KEYPC=m
CONFIG_SPEAKUP_SYNTH_LTLK=m
CONFIG_SPEAKUP_SYNTH_SOFT=m
CONFIG_SPEAKUP_SYNTH_SPKOUT=m
CONFIG_SPEAKUP_SYNTH_TXPRT=m
CONFIG_SPEAKUP_SYNTH_DUMMY=m
CONFIG_TOUCHSCREEN_CLEARPAD_TM1217=m
CONFIG_TOUCHSCREEN_SYNAPTICS_I2C_RMI4=m
CONFIG_DRM_PSB=m

#
# Altera FPGA firmware download module
#
# CONFIG_ALTERA_STAPL is not set
CONFIG_X86_PLATFORM_DEVICES=y
CONFIG_ACER_WMI=y
CONFIG_ACERHDF=y
CONFIG_DELL_LAPTOP=m
CONFIG_DELL_WMI=y
CONFIG_DELL_WMI_AIO=y
CONFIG_FUJITSU_LAPTOP=y
# CONFIG_FUJITSU_LAPTOP_DEBUG is not set
CONFIG_TC1100_WMI=y
CONFIG_HP_ACCEL=y
CONFIG_HP_WMI=y
CONFIG_MSI_LAPTOP=y
CONFIG_PANASONIC_LAPTOP=y
CONFIG_COMPAL_LAPTOP=y
CONFIG_SONY_LAPTOP=y
CONFIG_SONYPI_COMPAT=y
CONFIG_IDEAPAD_LAPTOP=y
CONFIG_THINKPAD_ACPI=y
# CONFIG_THINKPAD_ACPI_DEBUGFACILITIES is not set
# CONFIG_THINKPAD_ACPI_DEBUG is not set
CONFIG_THINKPAD_ACPI_UNSAFE_LEDS=y
CONFIG_THINKPAD_ACPI_VIDEO=y
CONFIG_THINKPAD_ACPI_HOTKEY_POLL=y
CONFIG_SENSORS_HDAPS=y
CONFIG_INTEL_MENLOW=y
CONFIG_EEEPC_LAPTOP=y
CONFIG_ASUS_WMI=y
CONFIG_ASUS_NB_WMI=y
CONFIG_EEEPC_WMI=y
CONFIG_ACPI_WMI=y
CONFIG_MSI_WMI=y
CONFIG_ACPI_ASUS=y
CONFIG_TOPSTAR_LAPTOP=y
CONFIG_ACPI_TOSHIBA=y
CONFIG_TOSHIBA_BT_RFKILL=y
CONFIG_ACPI_CMPC=y
CONFIG_INTEL_IPS=y
CONFIG_IBM_RTL=y
CONFIG_XO1_RFKILL=y
CONFIG_XO15_EBOOK=y
CONFIG_SAMSUNG_LAPTOP=y

#
# Firmware Drivers
#
CONFIG_EDD=y
# CONFIG_EDD_OFF is not set
CONFIG_FIRMWARE_MEMMAP=y
# CONFIG_EFI_VARS is not set
CONFIG_DELL_RBU=m
CONFIG_DCDBAS=m
CONFIG_DMIID=y
CONFIG_DMI_SYSFS=y
CONFIG_ISCSI_IBFT_FIND=y
CONFIG_ISCSI_IBFT=y
# CONFIG_SIGMA is not set

#
# File systems
#
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
CONFIG_EXT2_FS_XIP=y
CONFIG_EXT3_FS=y
CONFIG_EXT3_DEFAULTS_TO_ORDERED=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_XATTR=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
# CONFIG_EXT4_DEBUG is not set
CONFIG_FS_XIP=y
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
CONFIG_JBD2=y
# CONFIG_JBD2_DEBUG is not set
CONFIG_FS_MBCACHE=y
CONFIG_REISERFS_FS=y
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
CONFIG_REISERFS_FS_XATTR=y
CONFIG_REISERFS_FS_POSIX_ACL=y
CONFIG_REISERFS_FS_SECURITY=y
CONFIG_JFS_FS=y
CONFIG_JFS_POSIX_ACL=y
CONFIG_JFS_SECURITY=y
# CONFIG_JFS_DEBUG is not set
# CONFIG_JFS_STATISTICS is not set
CONFIG_XFS_FS=y
CONFIG_XFS_QUOTA=y
CONFIG_XFS_POSIX_ACL=y
CONFIG_XFS_RT=y
# CONFIG_XFS_DEBUG is not set
CONFIG_GFS2_FS=y
CONFIG_GFS2_FS_LOCKING_DLM=y
CONFIG_OCFS2_FS=y
CONFIG_OCFS2_FS_O2CB=y
CONFIG_OCFS2_FS_USERSPACE_CLUSTER=y
# CONFIG_OCFS2_FS_STATS is not set
CONFIG_OCFS2_DEBUG_MASKLOG=y
# CONFIG_OCFS2_DEBUG_FS is not set
CONFIG_BTRFS_FS=y
CONFIG_BTRFS_FS_POSIX_ACL=y
CONFIG_NILFS2_FS=y
CONFIG_FS_POSIX_ACL=y
CONFIG_EXPORTFS=y
CONFIG_FILE_LOCKING=y
CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY_USER=y
CONFIG_FANOTIFY=y
# CONFIG_FANOTIFY_ACCESS_PERMISSIONS is not set
CONFIG_QUOTA=y
CONFIG_QUOTA_NETLINK_INTERFACE=y
# CONFIG_PRINT_QUOTA_WARNING is not set
# CONFIG_QUOTA_DEBUG is not set
CONFIG_QUOTA_TREE=y
CONFIG_QFMT_V1=y
CONFIG_QFMT_V2=y
CONFIG_QUOTACTL=y
CONFIG_AUTOFS4_FS=y
CONFIG_FUSE_FS=y
CONFIG_CUSE=y
CONFIG_GENERIC_ACL=y

#
# Caches
#
CONFIG_FSCACHE=y
CONFIG_FSCACHE_STATS=y
CONFIG_FSCACHE_HISTOGRAM=y
# CONFIG_FSCACHE_DEBUG is not set
# CONFIG_FSCACHE_OBJECT_LIST is not set
CONFIG_CACHEFILES=y
# CONFIG_CACHEFILES_DEBUG is not set
CONFIG_CACHEFILES_HISTOGRAM=y

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=y
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_NTFS_FS=y
# CONFIG_NTFS_DEBUG is not set
CONFIG_NTFS_RW=y

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_CONFIGFS_FS=y
CONFIG_MISC_FILESYSTEMS=y
CONFIG_ADFS_FS=y
CONFIG_ADFS_FS_RW=y
CONFIG_AFFS_FS=y
CONFIG_ECRYPT_FS=y
CONFIG_HFS_FS=y
CONFIG_HFSPLUS_FS=y
CONFIG_BEFS_FS=y
# CONFIG_BEFS_DEBUG is not set
CONFIG_BFS_FS=y
CONFIG_EFS_FS=y
# CONFIG_LOGFS is not set
CONFIG_CRAMFS=y
CONFIG_SQUASHFS=y
CONFIG_SQUASHFS_XATTR=y
# CONFIG_SQUASHFS_LZO is not set
# CONFIG_SQUASHFS_XZ is not set
CONFIG_SQUASHFS_EMBEDDED=y
CONFIG_SQUASHFS_FRAGMENT_CACHE_SIZE=3
CONFIG_VXFS_FS=y
CONFIG_MINIX_FS=y
CONFIG_OMFS_FS=y
CONFIG_HPFS_FS=y
CONFIG_QNX4FS_FS=y
CONFIG_ROMFS_FS=y
CONFIG_ROMFS_BACKED_BY_BLOCK=y
CONFIG_ROMFS_ON_BLOCK=y
CONFIG_PSTORE=y
CONFIG_SYSV_FS=y
CONFIG_UFS_FS=y
# CONFIG_UFS_FS_WRITE is not set
# CONFIG_UFS_DEBUG is not set
CONFIG_EXOFS_FS=y
# CONFIG_EXOFS_DEBUG is not set
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
CONFIG_NFS_V3_ACL=y
CONFIG_NFS_V4=y
CONFIG_NFS_V4_1=y
CONFIG_PNFS_FILE_LAYOUT=y
# CONFIG_ROOT_NFS is not set
CONFIG_NFS_FSCACHE=y
CONFIG_NFS_USE_LEGACY_DNS=y
CONFIG_NFS_USE_NEW_IDMAPPER=y
CONFIG_NFSD=y
CONFIG_NFSD_DEPRECATED=y
CONFIG_NFSD_V2_ACL=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V3_ACL=y
CONFIG_NFSD_V4=y
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_NFS_ACL_SUPPORT=y
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=y
CONFIG_SUNRPC_GSS=y
CONFIG_SUNRPC_XPRT_RDMA=m
CONFIG_RPCSEC_GSS_KRB5=m
CONFIG_CEPH_FS=y
CONFIG_CIFS=y
CONFIG_CIFS_STATS=y
CONFIG_CIFS_STATS2=y
CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_UPCALL=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
# CONFIG_CIFS_DEBUG2 is not set
CONFIG_CIFS_DFS_UPCALL=y
CONFIG_CIFS_FSCACHE=y
CONFIG_CIFS_ACL=y
CONFIG_CIFS_EXPERIMENTAL=y
CONFIG_NCP_FS=y
CONFIG_NCPFS_PACKET_SIGNING=y
CONFIG_NCPFS_IOCTL_LOCKING=y
CONFIG_NCPFS_STRONG=y
CONFIG_NCPFS_NFS_NS=y
CONFIG_NCPFS_OS2_NS=y
CONFIG_NCPFS_SMALLDOS=y
CONFIG_NCPFS_NLS=y
CONFIG_NCPFS_EXTRAS=y
CONFIG_CODA_FS=y
CONFIG_AFS_FS=y
# CONFIG_AFS_DEBUG is not set
CONFIG_AFS_FSCACHE=y
CONFIG_9P_FS=m
CONFIG_9P_FSCACHE=y
CONFIG_9P_FS_POSIX_ACL=y

#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
CONFIG_ACORN_PARTITION=y
CONFIG_ACORN_PARTITION_CUMANA=y
CONFIG_ACORN_PARTITION_EESOX=y
CONFIG_ACORN_PARTITION_ICS=y
CONFIG_ACORN_PARTITION_ADFS=y
CONFIG_ACORN_PARTITION_POWERTEC=y
CONFIG_ACORN_PARTITION_RISCIX=y
CONFIG_OSF_PARTITION=y
CONFIG_AMIGA_PARTITION=y
CONFIG_ATARI_PARTITION=y
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
CONFIG_MINIX_SUBPARTITION=y
CONFIG_SOLARIS_X86_PARTITION=y
CONFIG_UNIXWARE_DISKLABEL=y
CONFIG_LDM_PARTITION=y
CONFIG_LDM_DEBUG=y
CONFIG_SGI_PARTITION=y
CONFIG_ULTRIX_PARTITION=y
CONFIG_SUN_PARTITION=y
CONFIG_KARMA_PARTITION=y
CONFIG_EFI_PARTITION=y
CONFIG_SYSV68_PARTITION=y
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="utf8"
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_CODEPAGE_737=m
CONFIG_NLS_CODEPAGE_775=m
CONFIG_NLS_CODEPAGE_850=y
CONFIG_NLS_CODEPAGE_852=y
CONFIG_NLS_CODEPAGE_855=m
CONFIG_NLS_CODEPAGE_857=m
CONFIG_NLS_CODEPAGE_860=y
CONFIG_NLS_CODEPAGE_861=m
# CONFIG_NLS_CODEPAGE_862 is not set
CONFIG_NLS_CODEPAGE_863=m
CONFIG_NLS_CODEPAGE_864=y
CONFIG_NLS_CODEPAGE_865=m
CONFIG_NLS_CODEPAGE_866=y
CONFIG_NLS_CODEPAGE_869=m
CONFIG_NLS_CODEPAGE_936=y
CONFIG_NLS_CODEPAGE_950=m
CONFIG_NLS_CODEPAGE_932=y
CONFIG_NLS_CODEPAGE_949=m
CONFIG_NLS_CODEPAGE_874=m
# CONFIG_NLS_ISO8859_8 is not set
CONFIG_NLS_CODEPAGE_1250=m
CONFIG_NLS_CODEPAGE_1251=m
CONFIG_NLS_ASCII=m
CONFIG_NLS_ISO8859_1=y
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=m
CONFIG_NLS_ISO8859_4=m
CONFIG_NLS_ISO8859_5=y
CONFIG_NLS_ISO8859_6=y
CONFIG_NLS_ISO8859_7=m
CONFIG_NLS_ISO8859_9=m
CONFIG_NLS_ISO8859_13=m
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=y
CONFIG_NLS_KOI8_R=y
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_UTF8=y
CONFIG_DLM=y
# CONFIG_DLM_DEBUG is not set

4,539 lines not shown · raw message

Part of this message is not shown here. View the raw message.

Thomas Gleixner

2011-05-04 09:53 UTC #14 lore raw

Quoted 15 lines · Ingo Molnar

1415 if (!nr_sectors)
1416 return 0;
1417
1418 /* Test device or partition size, when known. */
1419 maxsector = i_size_read(bio->bi_bdev->bd_inode) >> 9; <==== [ **CRASH** ]
1420 if (maxsector) {
1421 sector_t sector = bio->bi_sector;
1422
1423 if (maxsector < nr_sectors || maxsector - nr_sectors < sector) {
bio->bi_bdev has become NULL?
I do not think the _cond_resched() was called, judging from stack contents. But we just had an IRQ:

Quoted 7 lines

[<c1d74030>] ? common_interrupt+0x30/0x40
So we might have raced with block IO IRQ queue-completion/submission activites.
But maybe it was a reschedule after all, just the stack does not carry any traces of it anymore. IRQs do not clear ->bi_bdev, right? Unless the bio refcounts are wrong and an IRQ's completion actually frees the bio, right?

Looking at the call chain that's impossible:

generic_make_request
submit_bio
submit_bh

submit_bh does:

bio = bio_alloc()
bio_get(bio)
submit_bio(bio)
bio_put(bio)

So that bio is not yet known to anything else than the calling code.

One possibility is that bh->bdev is NULL when submit_bh() is called, which I think is rather unlikely, but can be easily verified with

fs/buffer.c +1 -0

@@ -2887,6 +2887,7 @@ int submit_bh(int rw, struct buffer_head * bh)
 	BUG_ON(!bh->b_end_io);
 	BUG_ON(buffer_delay(bh));
 	BUG_ON(buffer_unwritten(bh));
+	BUG_ON(!bh->b_bdev);
 
 	/*
 	 * Only clear out a write error when rewriting

But I rather suspect, that CONFIG_SLUB=y is the thing we need to look at. The lockless fastpath cmpxchg comes to my mind.

Either we generate broken code with that ELAN caused options or that combo triggers some hidden problem in SLUB.

Thanks,

tglx

Ingo Molnar

2011-05-04 10:13 UTC #81 lore raw

Quoted 2 lines · Ingo Molnar

I've built a CONFIG_DEBUG_PAGEALLOC=y and CONFIG_SLUB_DEBUG=y kernel, maybe the crash triggers in a more revealing way.

After an hour of testing it still has not triggered. This could have multiple reasons:

- Maybe i reproduced the original crash by pure luck.

To exclude this i'll re-test once again with the non-debug .config.

- There's some kernel image layout dependency that twiddling these options has hidden.

This is the lowest probability explanation, because the problem triggered for me after tweaking the original .config and using a different toolchain.

- These debug options could have narrowed the race window and could make the race much less likely to occur.

This is the highest probability explanation.

Thanks,

Ingo

Ingo Molnar

2011-05-04 10:19 UTC #15 lore raw

Quoted 2 lines · Thomas Gleixner

But I rather suspect, that CONFIG_SLUB=y is the thing we need to look at. The lockless fastpath cmpxchg comes to my mind.

Hm, and CONFIG_X86_ELAN, as Linus noted, has an impact on the cmpxchg implementation.

Quoted 2 lines

Either we generate broken code with that ELAN caused options or that combo triggers some hidden problem in SLUB.

Note that the crash went away with SLUB_DEBUG=y and PAGEALLOC=y and SLUB_DEBUG=y would certainly narrow any lockless-SLUB race windows.

Thanks,

Ingo

Ingo Molnar

2011-05-04 10:25 UTC #16 lore raw

Quoted 7 lines · Ingo Molnar

Quoted 2 lines · Thomas Gleixner
But I rather suspect, that CONFIG_SLUB=y is the thing we need to look at. The lockless fastpath cmpxchg comes to my mind.
Hm, and CONFIG_X86_ELAN, as Linus noted, has an impact on the cmpxchg implementation.

Walter, could you please test the patch below on a failing kernel?

Note, the patch is actually incorrect for a real Elan box, but should work on Walter's box - and should avoid the cmpxchg8b_emul implementation on that box.

Thanks,

Ingo

arch/x86/Kconfig.cpu +1 -1

index 6a7cfdf..0783906 100644
@@ -402,7 +402,7 @@ config X86_TSC
 
 config X86_CMPXCHG64
 	def_bool y
-	depends on X86_PAE || X86_64 || MCORE2 || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || MATOM
+	depends on X86_PAE || X86_64 || MCORE2 || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || MATOM || X86_ELAN
 
 # this should be set for all -march=.. options where the compiler
 # generates cmov.

Ingo Molnar

2011-05-04 10:33 UTC #17 lore raw

Quoted 1 line · Ingo Molnar

Walter, could you please test the patch below on a failing kernel?

the other patch to test would be the one below, on a failing kernel. This would check whether compiler flags have an impact on your crash.

Ingo

arch/x86/Makefile_32.cpu +0 -3

index f2ee1ab..58dae4f 100644
@@ -36,9 +36,6 @@ cflags-$(CONFIG_MCORE2)		+= -march=i686 $(call tune,core2)
 cflags-$(CONFIG_MATOM)		+= $(call cc-option,-march=atom,$(call cc-option,-march=core2,-march=i686)) \
 	$(call cc-option,-mtune=atom,$(call cc-option,-mtune=generic))
 
-# AMD Elan support
-cflags-$(CONFIG_X86_ELAN)	+= -march=i486
-
 # Geode GX1 support
 cflags-$(CONFIG_MGEODEGX1)	+= -march=pentium-mmx
 cflags-$(CONFIG_MGEODE_LX)	+= $(call cc-option,-march=geode,-march=pentium-mmx)

Ingo Molnar

2011-05-04 10:41 UTC #82 lore raw

Quoted 3 lines · Ingo Molnar

- Maybe i reproduced the original crash by pure luck.
To exclude this i'll re-test once again with the non-debug .config.

I got a different problem this time around:

Fedora Core release 6 (Zod)
Kernel 2.6.39-rc5-i486-1sys+ on an i686

mercury login: ata1: lost interrupt (Status 0x58)
ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen ata1.00: failed command: WRITE DMA EXT
ata1.00: cmd 35/00:38:56:cf:a5/00:01:05:00:00/e0 tag 0 dma 159744 out

         res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)

ata1.00: status: { DRDY }
ata1: soft resetting link
ata1: nv_mode_filter: 0x3f39f&0x3f39f->0x3f39f, BIOS=0x3f000 (0xc60000c0) ACPI=0x3f01f (20:600:0x13) ata1.00: configured for UDMA/100
ata1.00: device reported invalid CHS sector 0
ata1: EH complete
ata1: lost interrupt (Status 0x58)
ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
ata1.00: failed command: WRITE DMA EXT
ata1.00: cmd 35/00:38:56:cf:a5/00:01:05:00:00/e0 tag 0 dma 159744 out

         res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)

Fedora Core release 6 (Zod)
Kernel 2.6.39-rc5-i486-1sys+ on an i686

mercury login:
Fedora Core release 6 (Zod)
Kernel 2.6.39-rc5-i486-1sys+ on an i686

         res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)

The system is dead (is not performing any IO) after that.

Note that i reported similar problems early during this merge window, in this mail:

[sporadic crash] blk: request botched

That sha1 was 89078d572eb9ce8d4c04264b8b0ba86de0d74c8f - that already had lockless SLUB:

e8c500c2b64b: Merge branch 'slub/lockless' into for-linus

I never managed to pin that down, these IO problems were very sporadic in my test setup and never reproducible, until today.

Ingo

Ingo Molnar

2011-05-04 10:45 UTC #83 lore raw

Quoted 11 lines · Ingo Molnar

Note that i reported similar problems early during this merge window, in this mail:
[sporadic crash] blk: request botched
That sha1 was 89078d572eb9ce8d4c04264b8b0ba86de0d74c8f - that already had lockless SLUB:
e8c500c2b64b: Merge branch 'slub/lockless' into for-linus
I never managed to pin that down, these IO problems were very sporadic in my test setup and never reproducible, until today.

So i'm doing a testrun now with a failing kernel with the patch below applied, to disable the SLUB lockless code.

Ingo

mm/slub.c +2 -0

index 94d2a33..27bc3be 100644
@@ -30,6 +30,8 @@
 
 #include <trace/events/kmem.h>
 
+#undef CONFIG_CMPXCHG_LOCAL
+
 /*
  * Lock order:
  *   1. slab_lock(page)

Ingo Molnar

2011-05-04 11:07 UTC #84 lore raw

mm/slub.c line 37

Quoted 13 lines · Ingo Molnar

mm/slub.c +2 -0

index 94d2a33..27bc3be 100644
@@ -30,6 +30,8 @@
 
 #include <trace/events/kmem.h>
 
+#undef CONFIG_CMPXCHG_LOCAL
+
 /*
  * Lock order:
  *   1. slab_lock(page)

This seems rock solid after half an hour of testing. I'll keep it running longer, i still have no good data for how frequently the crashes are occuring.

Thanks,

Ingo

Thomas Gleixner

2011-05-04 11:11 UTC #20 lore raw

Quoted 8 lines · Ingo Molnar

Quoted 2 lines · Thomas Gleixner
But I rather suspect, that CONFIG_SLUB=y is the thing we need to look at. The lockless fastpath cmpxchg comes to my mind.
Hm, and CONFIG_X86_ELAN, as Linus noted, has an impact on the cmpxchg implementation.

Exactly. With ELAN CONFIG_X86_CMPXCHG64 is not set. When I disable ELAN it's set.

Quoted 5 lines

Quoted 2 lines
Either we generate broken code with that ELAN caused options or that combo triggers some hidden problem in SLUB.
Note that the crash went away with SLUB_DEBUG=y and PAGEALLOC=y and SLUB_DEBUG=y would certainly narrow any lockless-SLUB race windows.

Well, it's pretty simple:

CONFIG_X86_CMPXCHG64=y compiles this_cpu_generic_cmpxchg_double() into:

    28f7:       89 f9                   mov    %edi,%ecx
    28f9:       8b 3e                   mov    (%esi),%edi
    28fb:       89 45 e4                mov    %eax,-0x1c(%ebp)
    28fe:       89 c3                   mov    %eax,%ebx
    2900:       8b 45 f0                mov    -0x10(%ebp),%eax
    2903:       64 0f c7 0f             cmpxchg8b %fs:(%edi)
    2907:       0f 94 c0                sete   %al
    290a:       84 c0                   test   %al,%al
    290c:       88 45 e4                mov    %al,-0x1c(%ebp)
    290f:       74 a4                   je     28b5 <kmem_cache_alloc+0x29>

while CONFIG_X86_CMPXCHG64=n results in:

    28b0:       8b 03                   mov    (%ebx),%eax
    28b2:       64 8b 30                mov    %fs:(%eax),%esi
    28b5:       39 d6                   cmp    %edx,%esi
    28b7:       75 d2                   jne    288b <kmem_cache_alloc+0x26>
    28b9:       64 8b 50 04             mov    %fs:0x4(%eax),%edx
    28bd:       39 ca                   cmp    %ecx,%edx
    28bf:       75 ca                   jne    288b <kmem_cache_alloc+0x26>
    28c1:       8b 4b 14                mov    0x14(%ebx),%ecx
    28c4:       8b 0c 0e                mov    (%esi,%ecx,1),%ecx
    28c7:       64 89 08                mov    %ecx,%fs:(%eax)
    28ca:       8b 03                   mov    (%ebx),%eax
    28cc:       42                      inc    %edx
    28cd:       64 89 50 04             mov    %edx,%fs:0x4(%eax)

And that code runs with preemption enabled. So when the task gets preempted _BEFORE_ it has actuallty written back the data, then the race window is wide open.

I'm still trying to understand that macro hell which actually generates that code. I always thought that George Anzingers macro maze was horrible, but that's even worse.

Thanks,

tglx

Pekka Enberg

2011-05-04 11:17 UTC #21 lore raw

Quoted 51 lines · Thomas Gleixner

Quoted 8 lines · Ingo Molnar
Quoted 2 lines · Thomas Gleixner
But I rather suspect, that CONFIG_SLUB=y is the thing we need to look at. The lockless fastpath cmpxchg comes to my mind.
Hm, and CONFIG_X86_ELAN, as Linus noted, has an impact on the cmpxchg implementation.
Exactly. With ELAN CONFIG_X86_CMPXCHG64 is not set. When I disable ELAN it's set.
Quoted 5 lines
Quoted 2 lines
Either we generate broken code with that ELAN caused options or that combo triggers some hidden problem in SLUB.
Note that the crash went away with SLUB_DEBUG=y and PAGEALLOC=y and SLUB_DEBUG=y would certainly narrow any lockless-SLUB race windows.
Well, it's pretty simple:
CONFIG_X86_CMPXCHG64=y compiles this_cpu_generic_cmpxchg_double() into:
28f7: 89 f9 mov %edi,%ecx
28f9: 8b 3e mov (%esi),%edi 28fb: 89 45 e4 mov %eax,-0x1c(%ebp) 28fe: 89 c3 mov %eax,%ebx
2900: 8b 45 f0 mov -0x10(%ebp),%eax 2903: 64 0f c7 0f cmpxchg8b %fs:(%edi) 2907: 0f 94 c0 sete %al
290a: 84 c0 test %al,%al
290c: 88 45 e4 mov %al,-0x1c(%ebp) 290f: 74 a4 je 28b5 <kmem_cache_alloc+0x29>
while CONFIG_X86_CMPXCHG64=n results in:
28b0: 8b 03 mov (%ebx),%eax 28b2: 64 8b 30 mov %fs:(%eax),%esi 28b5: 39 d6 cmp %edx,%esi
28b7: 75 d2 jne 288b <kmem_cache_alloc+0x26> 28b9: 64 8b 50 04 mov %fs:0x4(%eax),%edx 28bd: 39 ca cmp %ecx,%edx
28bf: 75 ca jne 288b <kmem_cache_alloc+0x26> 28c1: 8b 4b 14 mov 0x14(%ebx),%ecx 28c4: 8b 0c 0e mov (%esi,%ecx,1),%ecx 28c7: 64 89 08 mov %ecx,%fs:(%eax) 28ca: 8b 03 mov (%ebx),%eax 28cc: 42 inc %edx
28cd: 64 89 50 04 mov %edx,%fs:0x4(%eax)

Quoted 7 lines

And that code runs with preemption enabled. So when the task gets preempted _BEFORE_ it has actuallty written back the data, then the race window is wide open.
I'm still trying to understand that macro hell which actually generates that code. I always thought that George Anzingers macro maze was horrible, but that's even worse.

I'm adding Christoph and Tejun to CC.

Tejun Heo

maintainer

2011-05-04 11:27 UTC #22 lore raw

Hello,

Quoted 33 lines · Pekka Enberg

Quoted 32 lines · Thomas Gleixner
On Wed, 4 May 2011, Ingo Molnar wrote:
Well, it's pretty simple:
CONFIG_X86_CMPXCHG64=y compiles this_cpu_generic_cmpxchg_double() into:
28f7: 89 f9 mov %edi,%ecx
28f9: 8b 3e mov (%esi),%edi 28fb: 89 45 e4 mov %eax,-0x1c(%ebp) 28fe: 89 c3 mov %eax,%ebx
2900: 8b 45 f0 mov -0x10(%ebp),%eax 2903: 64 0f c7 0f cmpxchg8b %fs:(%edi) 2907: 0f 94 c0 sete %al
290a: 84 c0 test %al,%al
290c: 88 45 e4 mov %al,-0x1c(%ebp) 290f: 74 a4 je 28b5 <kmem_cache_alloc+0x29>
while CONFIG_X86_CMPXCHG64=n results in:
28b0: 8b 03 mov (%ebx),%eax 28b2: 64 8b 30 mov %fs:(%eax),%esi 28b5: 39 d6 cmp %edx,%esi
28b7: 75 d2 jne 288b <kmem_cache_alloc+0x26> 28b9: 64 8b 50 04 mov %fs:0x4(%eax),%edx 28bd: 39 ca cmp %ecx,%edx
28bf: 75 ca jne 288b <kmem_cache_alloc+0x26> 28c1: 8b 4b 14 mov 0x14(%ebx),%ecx 28c4: 8b 0c 0e mov (%esi,%ecx,1),%ecx 28c7: 64 89 08 mov %ecx,%fs:(%eax) 28ca: 8b 03 mov (%ebx),%eax 28cc: 42 inc %edx
28cd: 64 89 50 04 mov %edx,%fs:0x4(%eax)

Quoted 3 lines

Quoted 3 lines
And that code runs with preemption enabled. So when the task gets preempted _BEFORE_ it has actuallty written back the data, then the race window is wide open.

Hmmm... if it's a race caused by preemtion enabled where it shouldn't be, it's most likely the wrong type of this_cpu_cmpxchg_double() being used in SLUB? ie. __this_cpu_cmpxchg_double() where it should have been this_cpu_cmpxchg_double()? Christoph?

Thanks.

Signature · 1 line

tejun

Ingo Molnar

2011-05-04 12:36 UTC #19 lore raw

Quoted 5 lines · Ingo Molnar

Quoted 2 lines · Ingo Molnar
* Thomas Gleixner <[email protected]> wrote:

Quoted 7 lines

Quoted 5 lines
Quoted 2 lines
But I rather suspect, that CONFIG_SLUB=y is the thing we need to look at. The lockless fastpath cmpxchg comes to my mind.
Hm, and CONFIG_X86_ELAN, as Linus noted, has an impact on the cmpxchg implementation.
Walter, could you please test the patch below on a failing kernel?

This patch will likely make the bug go away.

Ingo

Ingo Molnar

2011-05-04 12:37 UTC #18 lore raw

Quoted 7 lines · Ingo Molnar

Quoted 1 line · Ingo Molnar
Walter, could you please test the patch below on a failing kernel?
the other patch to test would be the one below, on a failing kernel. This would check whether compiler flags have an impact on your crash.

this patch will probably have no effect on the crash.

Thanks,

Ingo

Ingo Molnar

2011-05-04 12:38 UTC #85 lore raw

Quoted 11 lines · Ingo Molnar

Quoted 8 lines · Ingo Molnar

mm/slub.c +0 -0

index 94d2a33..27bc3be 100644
@@ -30,6 +30,8 @@
 
 #include <trace/events/kmem.h>

Quoted 8 lines

mm/slub.c line 37
Quoted 5 lines
mm/slub.c +2 -0
+#undef CONFIG_CMPXCHG_LOCAL
+
 /*
  * Lock order:
  *   1. slab_lock(page)
This seems rock solid after half an hour of testing. I'll keep it running longer, i still have no good data for how frequently the crashes are occuring.

It's still rock solid after 2 hours: neither crashes nor IO/IRQ timeouts are occuring.

Thanks,

Ingo

Ingo Molnar

2011-05-04 12:48 UTC #86 lore raw

Quoted 16 lines · Ingo Molnar

Quoted 7 lines
Quoted 7 lines
index 94d2a33..27bc3be 100644
mm/slub.c +0 -0
@@ -30,6 +30,8 @@
 
 #include <trace/events/kmem.h>
 
Quoted 8 lines
mm/slub.c line 37
Quoted 5 lines
mm/slub.c +2 -0
+#undef CONFIG_CMPXCHG_LOCAL
+
 /*
  * Lock order:
  *   1. slab_lock(page)
This seems rock solid after half an hour of testing. I'll keep it running longer, i still have no good data for how frequently the crashes are occuring.

Quoted 2 lines

It's still rock solid after 2 hours: neither crashes nor IO/IRQ timeouts are occuring.

So i removed the above patch and rebooted, and within minutes of starting the FS test i got:

skb_over_panic: text:c19fe045 len:98 put:98 head: (null) data: (null) tail:0x62 end:0x0 dev:<NULL>

------------[ cut here ]------------
kernel BUG at net/core/skbuff.c:127!
invalid opcode: 0000 [#1] SMP 
last sysfs file: /sys/devices/pci0000:00/0000:00:0a.0/net/eth0/address
Modules linked in:

Pid: 3535, comm: dd Not tainted 2.6.39-rc5-i486-1sys+ #122586 System manufacturer System Product Name/A8N-E
EIP: 0060:[<c1bda60d>] EFLAGS: 00010292 CPU: 1
EIP is at skb_put+0x89/0x92
EAX: 0000006b EBX: 00000000 ECX: 00000046 EDX: 00000000
ESI: c19fe045 EDI: 00000062 EBP: f64cdf20 ESP: f64cdef4
DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
Process dd (pid: 3535, ti=f64cc000 task=f5f4b570 task.ti=f53f4000)

Stack:
 c2143545 c19fe045 00000062 00000062 00000000 00000000 00000062 00000000
 c207d136 f6506000 f408d600 f64cdf4c c19fe045 c19fd92b f64cdf4c 00000040
 f6506428 00000000 34020062 f6506000 00000246 c21b799c f64cdf90 c1a004c1
Call Trace:
 [<c19fe045>] ? nv_rx_process_optimized+0x101/0x1de
 [<c19fe045>] nv_rx_process_optimized+0x101/0x1de
 [<c19fd92b>] ? nv_alloc_rx_optimized+0xe/0x18f
 [<c1a004c1>] nv_napi_poll+0x496/0x4a5
 [<c105838c>] ? hrtimer_run_pending+0xe/0xd1
 [<c1d734b4>] ? _raw_spin_lock+0x8/0x1e
 [<c1be1d59>] net_rx_action+0x94/0x1ab
 [<c1042fcd>] __do_softirq+0x9f/0x14f
 [<c1042f2e>] ? remote_softirq_receive+0x33/0x33
 <IRQ> 
 [<c10431e7>] ? irq_exit+0x3a/0x43
 [<c10047ce>] ? do_IRQ+0x8c/0xa0
 [<c116366d>] ? __ext3_journal_dirty_metadata+0x1e/0x45
 [<c1054f23>] ? wake_up_bit+0x1c/0x20
 [<c10ec726>] ? __brelse+0xb/0x36
 [<c102ea1c>] ? __wake_up_common+0xe/0x62
 [<c1d74eb0>] ? common_interrupt+0x30/0x40
 [<c14fb1ea>] ? sha_transform+0x9a/0x1be
 [<c15ff44e>] ? extract_buf+0x50/0xe3
 [<c14fe7ab>] ? __copy_to_user_ll+0xb/0x37
 [<c14fe9b5>] ? copy_to_user+0x3e/0x49
 [<c15ffd83>] ? extract_entropy_user+0x80/0xe5
 [<c15ffdfa>] ? urandom_read+0x12/0x14
 [<c10cc888>] ? vfs_read+0x93/0x115
 [<c15ffde8>] ? extract_entropy_user+0xe5/0xe5
 [<c10cc94c>] ? sys_read+0x42/0x66
 [<c1d74903>] ? sysenter_do_call+0x12/0x28
Code: 00 00 89 44 24 14 8b 81 a8 00 00 00 89 44 24 10 89 54 24 0c 8b 41 50 89 44 24 08 89 74 24 04 c7 04 24 45 35 14 c2 e8 fa 09 18 00 <0f> 0b 83 c4 24 5b 5e 5d c3 55 89 e5 57 56 53 83 ec 30 e8 ac a8 
EIP: [<c1bda60d>] skb_put+0x89/0x92 SS:ESP 0068:f64cdef4
---[ end trace 1d38b9741c67ed6b ]---

And in hindsight i have to admit that i saw this in randconfig testing in the past few weeks, i just never managed to reproduce it ...

So yes, the fact that this time it crashed in networking (not in block IO) clearly implicates SLUB as well.

And the trigger condition is the lockless SLUB code on 32-bit, non-64-bit-cmpxchg platforms. I'd not be surprised if some embedded platforms triggered this too.

Ingo

Pekka Enberg

2011-05-04 12:51 UTC #23 lore raw

Hi Tejun,

Quoted 4 lines · Tejun Heo

Hmmm... if it's a race caused by preemtion enabled where it shouldn't be, it's most likely the wrong type of this_cpu_cmpxchg_double() being used in SLUB? ie. __this_cpu_cmpxchg_double() where it should have been this_cpu_cmpxchg_double()? Christoph?

There's no __this_cpu_cmpxchg_double() usage in mm/slub.c so I don't think it's that simple.

Ingo Molnar

2011-05-04 12:57 UTC #24 lore raw

Quoted 10 lines · Pekka Enberg

Hi Tejun,
Quoted 4 lines · Tejun Heo
Hmmm... if it's a race caused by preemtion enabled where it shouldn't be, it's most likely the wrong type of this_cpu_cmpxchg_double() being used in SLUB? ie. __this_cpu_cmpxchg_double() where it should have been this_cpu_cmpxchg_double()? Christoph?
There's no __this_cpu_cmpxchg_double() usage in mm/slub.c so I don't think it's that simple.

Well, AFAICS the problem is:

earth4:~/tip> grep cmpxchg mm/slub.c

		if (unlikely(!this_cpu_cmpxchg_double(
		if (unlikely(!this_cpu_cmpxchg_double(

Where this macro resolves to:

# define this_cpu_cmpxchg_double_8(pcp1, pcp2, oval1, oval2, nval1, nval2) \
_this_cpu_generic_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)

where:

#define _this_cpu_generic_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2) \

({                                                                      \
        int ret__;                                                      \
        preempt_disable();                                              \
        ret__ = __this_cpu_generic_cmpxchg_double(pcp1, pcp2,           \
                        oval1, oval2, nval1, nval2);                    \
        preempt_enable();

where:

#define __this_cpu_generic_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2) \

({                                                                      \
        int __ret = 0;                                                  \
        if (__this_cpu_read(pcp1) == (oval1) &&                         \
                         __this_cpu_read(pcp2)  == (oval2)) {           \
                __this_cpu_write(pcp1, (nval1));                        \
                __this_cpu_write(pcp2, (nval2));                        \
                __ret = 1;                                              \
        }                                                               \
        (__ret);                                                        \

})

With is both IRQ and SMP unsafe.

Thanks,

Ingo

Thomas Gleixner

2011-05-04 13:01 UTC #26 lore raw

Quoted 8 lines · Tejun Heo

Quoted 3 lines
Quoted 3 lines
And that code runs with preemption enabled. So when the task gets preempted _BEFORE_ it has actuallty written back the data, then the race window is wide open.
Hmmm... if it's a race caused by preemtion enabled where it shouldn't be, it's most likely the wrong type of this_cpu_cmpxchg_double() being used in SLUB? ie. __this_cpu_cmpxchg_double() where it should have been this_cpu_cmpxchg_double()? Christoph?

No, the problem is that ELAN prevents the cmpxchg8b, but keeps CONFIG_CMPXCHG_LOCAL=y which then results in the unprotected code for the following reason:

this_cpu_cmpxchg_double()

-> __pcpu_double_call_return_bool

-> this_cpu_cmpxchg_double_4

Which on x86 expands to

-> percpu_cmpxchg8b_double() when CONFIG_X86_CMPXCHG64=y

With CONFIG_X86_CMPXCHG64=n it expands to the default:

_this_cpu_generic_cmpxchg_double() in linux/percpu.h

#define _this_cpu_generic_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2) \

({                                                                      \
        int ret__;                                                      \
        preempt_disable();                                              \
        ret__ = __this_cpu_generic_cmpxchg_double(pcp1, pcp2,           \
                        oval1, oval2, nval1, nval2);                    \
        preempt_enable();                                               \
        ret__;                                                          \

})

And:

#define __this_cpu_generic_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2) \

({                                                                      \
        int __ret = 0;                                                  \
        if (__this_cpu_read(pcp1) == (oval1) &&                         \
                         __this_cpu_read(pcp2)  == (oval2)) {           \
                __this_cpu_write(pcp1, (nval1));                        \
                __this_cpu_write(pcp2, (nval2));                        \
                __ret = 1;                                              \
        }                                                               \
        (__ret);                                                        \

})

So now that failing config has CONFIG_PREEMPT=n which makes preempt_disable / enable a nop.

So preemption is not the problem, but what about interrupts and softirqs ?

So the question is whether CMPXCHG_LOCAL for x86 wants to depend on X86_CMPXCHG64.

The other solution is to use irqsafe_cpu_cmpxchg_double() instead of this_cpu_cmpxchg_double() in slub.c.

This will not hurt the X86_CMPXCHG64=y case, but keep the expansion to the above __this_cpu_generic_cmpxchg_double working.

Which makes me even wonder some more whether we need that whole CMPXCHG_LOCAL #ifdeffery in slub.c at all.

Thanks,

tglx

Thomas Gleixner

2011-05-04 13:03 UTC #25 lore raw

Quoted 46 lines · Ingo Molnar

Quoted 10 lines · Pekka Enberg
Hi Tejun,
Quoted 4 lines · Tejun Heo
Hmmm... if it's a race caused by preemtion enabled where it shouldn't be, it's most likely the wrong type of this_cpu_cmpxchg_double() being used in SLUB? ie. __this_cpu_cmpxchg_double() where it should have been this_cpu_cmpxchg_double()? Christoph?
There's no __this_cpu_cmpxchg_double() usage in mm/slub.c so I don't think it's that simple.
Well, AFAICS the problem is:
earth4:~/tip> grep cmpxchg mm/slub.c
		if (unlikely(!this_cpu_cmpxchg_double(
		if (unlikely(!this_cpu_cmpxchg_double(
Where this macro resolves to:
# define this_cpu_cmpxchg_double_8(pcp1, pcp2, oval1, oval2, nval1, nval2) \
_this_cpu_generic_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)
where:
#define _this_cpu_generic_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2) \
({                                                                      \
        int ret__;                                                      \
        preempt_disable();                                              \
        ret__ = __this_cpu_generic_cmpxchg_double(pcp1, pcp2,           \
                        oval1, oval2, nval1, nval2);                    \
        preempt_enable();                                              
where:
#define __this_cpu_generic_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2) \
({                                                                      \
        int __ret = 0;                                                  \
        if (__this_cpu_read(pcp1) == (oval1) &&                         \
                         __this_cpu_read(pcp2)  == (oval2)) {           \
                __this_cpu_write(pcp1, (nval1));                        \
                __this_cpu_write(pcp2, (nval2));                        \
                __ret = 1;                                              \
        }                                                               \
        (__ret);                                                        \

Quoted 3 lines

})
With is both IRQ and SMP unsafe.

SMP is not an issue because that's cpu local access, but it's irq/softirq unsafe. See my other mail.

Thanks,

tglx

Tejun Heo

maintainer

2011-05-04 13:20 UTC #27 lore raw

Hello,

Quoted 13 lines · Thomas Gleixner

Quoted 8 lines · Tejun Heo
Quoted 3 lines
Quoted 3 lines
And that code runs with preemption enabled. So when the task gets preempted _BEFORE_ it has actuallty written back the data, then the race window is wide open.
Hmmm... if it's a race caused by preemtion enabled where it shouldn't be, it's most likely the wrong type of this_cpu_cmpxchg_double() being used in SLUB? ie. __this_cpu_cmpxchg_double() where it should have been this_cpu_cmpxchg_double()? Christoph?
No, the problem is that ELAN prevents the cmpxchg8b, but keeps CONFIG_CMPXCHG_LOCAL=y which then results in the unprotected code for the following reason:

...

Quoted 5 lines

So the question is whether CMPXCHG_LOCAL for x86 wants to depend on X86_CMPXCHG64.
The other solution is to use irqsafe_cpu_cmpxchg_double() instead of this_cpu_cmpxchg_double() in slub.c.

I think this is the root cause. CMPXCHG_LOCAL is an optimization flag, indicating that the processor provides fast local cmpxchg, it doesn't say anything about local synchronization properties and if the code required irq exclusion, it should have used irqsafe_cpu_cmpxchg_double() whether the processor supports it natively or not, so there's the bug. Pekka, can you please change the offending cmpxchg_double() to irqsafe variant?

As for CMPXCHG_LOCAL being set spuriously, maybe introduce CMPXCHG_DOUBLE_LOCAL? I don't know. It's pretty nasty to implement different high-level code paths depending on CPU features. We can't even determine whether the feature will be actually available at compile time. But, then again, it might incur noticeable slowdown for cases where the generic implementation is used. Has anyone measured the difference against before the whole this_cpu conversion?

Thanks.

Signature · 1 line

tejun

Ingo Molnar

2011-05-04 13:23 UTC #76 lore raw

Quoted 2 lines · Thomas Gleixner

Which makes me even wonder some more whether we need that whole CMPXCHG_LOCAL #ifdeffery in slub.c at all.

I always found that #ifdeffery rather repulsive.

Thanks,

Ingo

Christoph Lameter

2011-05-04 14:04 UTC #78 lore raw

In reply to Pekka Enberg

Quoted 35 lines · Pekka Enberg

Quoted 35 lines
Well, it's pretty simple:
CONFIG_X86_CMPXCHG64=y compiles this_cpu_generic_cmpxchg_double() into:
28f7: 89 f9 mov %edi,%ecx
28f9: 8b 3e mov (%esi),%edi 28fb: 89 45 e4 mov %eax,-0x1c(%ebp) 28fe: 89 c3 mov %eax,%ebx
2900: 8b 45 f0 mov -0x10(%ebp),%eax 2903: 64 0f c7 0f cmpxchg8b %fs:(%edi) 2907: 0f 94 c0 sete %al
290a: 84 c0 test %al,%al
290c: 88 45 e4 mov %al,-0x1c(%ebp) 290f: 74 a4 je 28b5 <kmem_cache_alloc+0x29>
while CONFIG_X86_CMPXCHG64=n results in:
28b0: 8b 03 mov (%ebx),%eax 28b2: 64 8b 30 mov %fs:(%eax),%esi 28b5: 39 d6 cmp %edx,%esi
28b7: 75 d2 jne 288b <kmem_cache_alloc+0x26> 28b9: 64 8b 50 04 mov %fs:0x4(%eax),%edx 28bd: 39 ca cmp %ecx,%edx
28bf: 75 ca jne 288b <kmem_cache_alloc+0x26> 28c1: 8b 4b 14 mov 0x14(%ebx),%ecx 28c4: 8b 0c 0e mov (%esi,%ecx,1),%ecx 28c7: 64 89 08 mov %ecx,%fs:(%eax) 28ca: 8b 03 mov (%ebx),%eax 28cc: 42 inc %edx
28cd: 64 89 50 04 mov %edx,%fs:0x4(%eax)
And that code runs with preemption enabled. So when the task gets preempted _BEFORE_ it has actuallty written back the data, then the race window is wide open.

Quoted 5 lines

Quoted 3 lines
I'm still trying to understand that macro hell which actually generates that code. I always thought that George Anzingers macro maze was horrible, but that's even worse.
I'm adding Christoph and Tejun to CC.

The above should not happen. If a kernel config indicates that there is no cmpxchg16b/cmpxchg8b available then we need to not compile the path that uses cmpxchg8b/cmpxchg16b. Guess we need CMPXCHG_DOUBLE_LOCAL or so.

Tejun Heo

maintainer

2011-05-04 14:08 UTC #79 lore raw

Hello, Christoph.

Quoted 3 lines · Christoph Lameter

The above should not happen. If a kernel config indicates that there is no cmpxchg16b/cmpxchg8b available then we need to not compile the path that uses cmpxchg8b/cmpxchg16b. Guess we need CMPXCHG_DOUBLE_LOCAL or so.

But regardless of that, if the code requires irq safety it should be using irqsafe_* operations. CMPXCHG_LOCAL is an optimization hint and doesn't necessarily imply stronger synchronization guarantees. In addition, there's no downside to using irqsafe_* variant when CPU actually supports cmpxchg[_double]. So, let's first fix that and think about CMPXCHG[_DOUBLE]_LOCAL or whatnot later.

Thank you.

Signature · 1 line

tejun

Thomas Gleixner

2011-05-04 14:10 UTC #28 lore raw

Quoted 23 lines · Tejun Heo

Hello,
Quoted 13 lines · Thomas Gleixner
Quoted 8 lines · Tejun Heo
Quoted 3 lines
Quoted 3 lines
And that code runs with preemption enabled. So when the task gets preempted _BEFORE_ it has actuallty written back the data, then the race window is wide open.
Hmmm... if it's a race caused by preemtion enabled where it shouldn't be, it's most likely the wrong type of this_cpu_cmpxchg_double() being used in SLUB? ie. __this_cpu_cmpxchg_double() where it should have been this_cpu_cmpxchg_double()? Christoph?
No, the problem is that ELAN prevents the cmpxchg8b, but keeps CONFIG_CMPXCHG_LOCAL=y which then results in the unprotected code for the following reason:
...
Quoted 5 lines
So the question is whether CMPXCHG_LOCAL for x86 wants to depend on X86_CMPXCHG64.
The other solution is to use irqsafe_cpu_cmpxchg_double() instead of this_cpu_cmpxchg_double() in slub.c.

Quoted 7 lines

I think this is the root cause. CMPXCHG_LOCAL is an optimization flag, indicating that the processor provides fast local cmpxchg, it doesn't say anything about local synchronization properties and if the code required irq exclusion, it should have used irqsafe_cpu_cmpxchg_double() whether the processor supports it natively or not, so there's the bug. Pekka, can you please change the offending cmpxchg_double() to irqsafe variant?

Patch below. Ingo, can you test that please ?

Quoted 2 lines

As for CMPXCHG_LOCAL being set spuriously, maybe introduce CMPXCHG_DOUBLE_LOCAL? I don't know. It's pretty nasty to implement

Oh no, please not another CONFIG_WTF and more #ifdeffery.

Quoted 3 lines

different high-level code paths depending on CPU features. We can't even determine whether the feature will be actually available at compile time. But, then again, it might incur noticeable slowdown for

Right, and the x86 implementation should not do

#ifdef CONFIG_X86_CMPXCHG64
#define percpu_cmpxchg8b_double(pcp1, o1, o2, n1, n2) #endif

And let the code fallback to the generic variant. It should have an #else path using the the cmpxchg64_local() implementation which has alternatives for runtime selection of cmpxchg8b or the cli protected emulation.

Quoted 2 lines

cases where the generic implementation is used. Has anyone measured the difference against before the whole this_cpu conversion?

Yes, that really wants to be done. The whole CMPXCHG_LOCAL ifdeffery should have been avoided in the first place. this_cpu_cmpxchg can really be implemented with preempt_enable/disable and the irqsafe variant in any case.

Thanks,

tglx

--------->
Subject: slub-hmm.patch
From: Thomas Gleixner <[email protected]>
Date: Wed, 04 May 2011 15:38:19 +0200

Signed-off-by Thomas Gleixner <[email protected]>

---
 include/linux/percpu.h |    2 +-
 mm/slub.c              |    4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

include/linux/percpu.h +1 -1

Index: linux-2.6/include/linux/percpu.h
===================================================================
@@ -948,7 +948,7 @@ do {									\
 	irqsafe_generic_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)
 # endif
 # define irqsafe_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)	\
-	__pcpu_double_call_return_int(irqsafe_cpu_cmpxchg_double_, (pcp1), (pcp2), (oval1), (oval2), (nval1), (nval2))
+	__pcpu_double_call_return_bool(irqsafe_cpu_cmpxchg_double_, (pcp1), (pcp2), (oval1), (oval2), (nval1), (nval2))
 #endif
 
 #endif /* __LINUX_PERCPU_H */

mm/slub.c +2 -2

Index: linux-2.6/mm/slub.c
===================================================================
@@ -1940,7 +1940,7 @@ redo:
 		 * Since this is without lock semantics the protection is only against
 		 * code executing on this cpu *not* from access by other cpus.
 		 */
-		if (unlikely(!this_cpu_cmpxchg_double(
+		if (unlikely(!irqsafe_cpu_cmpxchg_double(
 				s->cpu_slab->freelist, s->cpu_slab->tid,
 				object, tid,
 				get_freepointer(s, object), next_tid(tid)))) {

@@ -2145,7 +2145,7 @@ redo:
 		set_freepointer(s, object, c->freelist);
 
 #ifdef CONFIG_CMPXCHG_LOCAL
-		if (unlikely(!this_cpu_cmpxchg_double(
+		if (unlikely(!irqsafe_cpu_cmpxchg_double(
 				s->cpu_slab->freelist, s->cpu_slab->tid,
 				c->freelist, tid,
 				object, next_tid(tid)))) {

Ingo Molnar

2011-05-04 14:14 UTC #29 lore raw

Quoted 1 line · Thomas Gleixner

Patch below. Ingo, can you test that please ?

Sure - test underway.

Thanks,

Ingo

Christoph Lameter

2011-05-04 14:19 UTC #36 lore raw

include/linux/percpu.h do { \ · line 954

Quoted 13 lines · Thomas Gleixner

Index: linux-2.6/include/linux/percpu.h
===================================================================

include/linux/percpu.h +1 -1

@@ -948,7 +948,7 @@ do {									\
 	irqsafe_generic_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)
 # endif
 # define irqsafe_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)	\
-	__pcpu_double_call_return_int(irqsafe_cpu_cmpxchg_double_, (pcp1), (pcp2), (oval1), (oval2), (nval1), (nval2))
+	__pcpu_double_call_return_bool(irqsafe_cpu_cmpxchg_double_, (pcp1), (pcp2), (oval1), (oval2), (nval1), (nval2))
 #endif
 
 #endif /* __LINUX_PERCPU_H */

Looking at the same thing here testing a similar patch. This should go separately as a bug fix.

Quoted 15 lines

Index: linux-2.6/mm/slub.c
===================================================================

mm/slub.c +1 -1

@@ -1940,7 +1940,7 @@ redo:
 		 * Since this is without lock semantics the protection is only against
 		 * code executing on this cpu *not* from access by other cpus.
 		 */
-		if (unlikely(!this_cpu_cmpxchg_double(
+		if (unlikely(!irqsafe_cpu_cmpxchg_double(
 				s->cpu_slab->freelist, s->cpu_slab->tid,
 				object, tid,
 				get_freepointer(s, object), next_tid(tid)))) {

mm/slub.c redo: · line 2151

Quoted 8 lines

mm/slub.c +1 -1

@@ -2145,7 +2145,7 @@ redo:
 		set_freepointer(s, object, c->freelist);
 
 #ifdef CONFIG_CMPXCHG_LOCAL
-		if (unlikely(!this_cpu_cmpxchg_double(
+		if (unlikely(!irqsafe_cpu_cmpxchg_double(
 				s->cpu_slab->freelist, s->cpu_slab->tid,
 				c->freelist, tid,
 				object, next_tid(tid)))) {

Ok as a basic fix since it does not change the code generated for the common x86 and other.

My version also takes out the CONFIG_CMPXCHG_LOCAL.

We could make the above two patches and then make the CONFIG_CMPXCHG_LOCAL removal a separate one (since it requires some benchmarking).

Christoph Lameter

2011-05-04 14:21 UTC #77 lore raw

Quoted 2 lines · Thomas Gleixner

So the question is whether CMPXCHG_LOCAL for x86 wants to depend on X86_CMPXCHG64.

Right.

Quoted 8 lines

The other solution is to use irqsafe_cpu_cmpxchg_double() instead of this_cpu_cmpxchg_double() in slub.c.
This will not hurt the X86_CMPXCHG64=y case, but keep the expansion to the above __this_cpu_generic_cmpxchg_double working.
Which makes me even wonder some more whether we need that whole CMPXCHG_LOCAL #ifdeffery in slub.c at all.

Hmmm... I have not measured it but it would certainly be nice to get rid of it. The period of irq disablement is then reduced in the fastpath too. However, we need to do additional tid checking (which should be fast).

The following patch does that and runs here in kvm

---
 include/linux/percpu.h   |    2 -
 include/linux/slub_def.h |    2 -
 mm/slub.c                |   60 +----------------------------------------------
 3 files changed, 3 insertions(+), 61 deletions(-)

include/linux/percpu.h +1 -1

Index: linux-2.6/include/linux/percpu.h
===================================================================
@@ -948,7 +948,7 @@ do {									\
 	irqsafe_generic_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)
 # endif
 # define irqsafe_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)	\
-	__pcpu_double_call_return_int(irqsafe_cpu_cmpxchg_double_, (pcp1), (pcp2), (oval1), (oval2), (nval1), (nval2))
+	__pcpu_double_call_return_bool(irqsafe_cpu_cmpxchg_double_, (pcp1), (pcp2), (oval1), (oval2), (nval1), (nval2))
 #endif
 
 #endif /* __LINUX_PERCPU_H */

include/linux/slub_def.h +0 -2

Index: linux-2.6/include/linux/slub_def.h
===================================================================
@@ -37,9 +37,7 @@ enum stat_item {
 
 struct kmem_cache_cpu {
 	void **freelist;	/* Pointer to next available object */
-#ifdef CONFIG_CMPXCHG_LOCAL
 	unsigned long tid;	/* Globally unique transaction id */
-#endif
 	struct page *page;	/* The slab from which we are allocating */
 	int node;		/* The node of the page (or -1 for debug) */
 #ifdef CONFIG_SLUB_STATS

mm/slub.c +2 ~~-58~~

213 lines not on this page · raw message

Thomas Gleixner

2011-05-04 14:21 UTC #80 lore raw

Quoted 4 lines · Christoph Lameter

The above should not happen. If a kernel config indicates that there is no cmpxchg16b/cmpxchg8b available then we need to not compile the path that uses cmpxchg8b/cmpxchg16b. Guess we need CMPXCHG_DOUBLE_LOCAL or so.

Oh no, you have perfectly fine replacements for that in percpu.h with irqsafe and preempt safe versions. So why adding another CONFIG from hell which creates even more ifdef mess ?

The whole CONFIG_CMPXCHG_LOCAL ifdeffery in slub.c is horrible and I think it's not necessary at all for two reasons:

1) irqsafe_cpu_cmpxchg() can be emulated cheap and we still have the advantatage of shorter irq disabled sections. And the difference between the current CONFIG_CMPXCHG_LOCAL=n and using the emulation macros with the short irqsave/restore around the access is probably not measurable at all.

2) that would actually test the code under all possible combinations and not splitting the tester base into x86 and !x86. So that bug would have been avoided in the first place.

Thanks,

tglx

Tejun Heo

maintainer

2011-05-04 14:25 UTC #37 lore raw

Hello,

Quoted 7 lines · Thomas Gleixner

Quoted 2 lines
cases where the generic implementation is used. Has anyone measured the difference against before the whole this_cpu conversion?
Yes, that really wants to be done. The whole CMPXCHG_LOCAL ifdeffery should have been avoided in the first place. this_cpu_cmpxchg can really be implemented with preempt_enable/disable and the irqsafe variant in any case.

Yeah, slub code looks pretty scary with the #ifdefs. IIUC, the problem was that cmpxchg_double is an optimization for fast path which was already very light weight and an extra locked op or irq on/off would have made considerable difference.

The cmpxchg_double optimization made the fast path go quite faster when CPU supports it but it may as well slow things down considerably if CPU doesn't, due to extra irq on/off's. Anyways, here's hoping that the slow down is acceptable compared to the base code without cmpxchg_double and the ugliness can be removed.

Thanks.

Signature · 1 line

tejun

Christoph Lameter

2011-05-04 14:35 UTC #38 lore raw

Quoted 5 lines · Tejun Heo

The cmpxchg_double optimization made the fast path go quite faster when CPU supports it but it may as well slow things down considerably if CPU doesn't, due to extra irq on/off's. Anyways, here's hoping that the slow down is acceptable compared to the base code without cmpxchg_double and the ugliness can be removed.

I think we are all in agreement. First path to fix the percpu macros:

Subject: percpu: Fix irqsafe_cpu_cmpxchg_double

The function in the macro was not updated when the function was given a bool return value.

Signed-off-by Christoph Lameter <[email protected]>

---
 include/linux/percpu.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

include/linux/percpu.h +1 -1

Index: linux-2.6/include/linux/percpu.h
===================================================================
@@ -948,7 +948,7 @@ do {									\
 	irqsafe_generic_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)
 # endif
 # define irqsafe_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)	\
-	__pcpu_double_call_return_int(irqsafe_cpu_cmpxchg_double_, (pcp1), (pcp2), (oval1), (oval2), (nval1), (nval2))
+	__pcpu_double_call_return_bool(irqsafe_cpu_cmpxchg_double_, (pcp1), (pcp2), (oval1), (oval2), (nval1), (nval2))
 #endif
 
 #endif /* __LINUX_PERCPU_H */

Ingo Molnar

2011-05-04 14:36 UTC #30 lore raw

PATCH 1/1 slub: Fix the lockless code on 32-bit platforms with no 64-bit cmpxchg

Link http://lkml.kernel.org/r/alpine.LFD.2.02.1105041539050.3005@ionos open here

Quoted 6 lines · Ingo Molnar

Quoted 1 line · Thomas Gleixner
Patch below. Ingo, can you test that please ?
Sure - test underway.

Ok, the patch below is looking really good - the test would have crashed on the bad kernel. I think we can consider the regression fixed:

Acked-and-tested-by Ingo Molnar <[email protected]>

I'll keep it running some more to make really sure it's fixed, plus it would be nice if Walter tested your fix as well.

Thanks,

Ingo

----------------->

Quoted 1 line

From c22bd309573330e33a77c329405d9473fc14f1cb Mon Sep 17 00:00:00 2001

From: Thomas Gleixner <[email protected]>
Date: Wed, 4 May 2011 15:38:19 +0200
Subject: [PATCH] slub: Fix the lockless code on 32-bit platforms with no 64-bit cmpxchg

Signed-off-by Thomas Gleixner <[email protected]>

Cc Pekka Enberg <[email protected]>

Cc Linus Torvalds <[email protected]>

Cc Jens Axboe <[email protected]>

Cc werner <[email protected]>

Cc Christoph Lameter <[email protected]>

Cc Tejun Heo <[email protected]>

Signed-off-by Ingo Molnar <[email protected]>

---
 include/linux/percpu.h |    2 +-
 mm/slub.c              |    4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

include/linux/percpu.h +1 -1

index 3a5c444..8b97308 100644
@@ -948,7 +948,7 @@ do {									\
 	irqsafe_generic_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)
 # endif
 # define irqsafe_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)	\
-	__pcpu_double_call_return_int(irqsafe_cpu_cmpxchg_double_, (pcp1), (pcp2), (oval1), (oval2), (nval1), (nval2))
+	__pcpu_double_call_return_bool(irqsafe_cpu_cmpxchg_double_, (pcp1), (pcp2), (oval1), (oval2), (nval1), (nval2))
 #endif
 
 #endif /* __LINUX_PERCPU_H */

mm/slub.c +2 -2

index 94d2a33..9d2e5e4 100644
@@ -1940,7 +1940,7 @@ redo:
 		 * Since this is without lock semantics the protection is only against
 		 * code executing on this cpu *not* from access by other cpus.
 		 */
-		if (unlikely(!this_cpu_cmpxchg_double(
+		if (unlikely(!irqsafe_cpu_cmpxchg_double(
 				s->cpu_slab->freelist, s->cpu_slab->tid,
 				object, tid,
 				get_freepointer(s, object), next_tid(tid)))) {

@@ -2145,7 +2145,7 @@ redo:
 		set_freepointer(s, object, c->freelist);
 
 #ifdef CONFIG_CMPXCHG_LOCAL
-		if (unlikely(!this_cpu_cmpxchg_double(
+		if (unlikely(!irqsafe_cpu_cmpxchg_double(
 				s->cpu_slab->freelist, s->cpu_slab->tid,
 				c->freelist, tid,
 				object, next_tid(tid)))) {

Christoph Lameter

Acked-by on v1 1/1

2011-05-04 14:42 UTC #31 lore raw

Re: [PATCH] slub: Fix the lockless code on 32-bit platforms with no 64-bit cmpxchg

Quoted 7 lines · Ingo Molnar

Ok, the patch below is looking really good - the test would have crashed on the bad kernel. I think we can consider the regression fixed:
Acked-and-tested-by Ingo Molnar <[email protected]>
I'll keep it running some more to make really sure it's fixed, plus it would be nice if Walter tested your fix as well.

Looks good and is exactly what I have here.

Acked-by Christoph Lameter <[email protected]>

Thomas Gleixner

2011-05-04 14:46 UTC #40 lore raw

Quoted 11 lines · Tejun Heo

Hello,
Quoted 7 lines · Thomas Gleixner
Quoted 2 lines
cases where the generic implementation is used. Has anyone measured the difference against before the whole this_cpu conversion?
Yes, that really wants to be done. The whole CMPXCHG_LOCAL ifdeffery should have been avoided in the first place. this_cpu_cmpxchg can really be implemented with preempt_enable/disable and the irqsafe variant in any case.

Quoted 8 lines

Yeah, slub code looks pretty scary with the #ifdefs. IIUC, the problem was that cmpxchg_double is an optimization for fast path which was already very light weight and an extra locked op or irq on/off would have made considerable difference.
The cmpxchg_double optimization made the fast path go quite faster when CPU supports it but it may as well slow things down considerably if CPU doesn't, due to extra irq on/off's. Anyways, here's hoping

Not really.

CMPXCHG_LOCAL=n

	local_irq_save();
	  handle_everything();
	local_irq_restore();

vs.

CMPXCHG_LOCAL=y

	do_prep();
	local_irq_save();
	  emulate_cmpxchg();
	local_irq_restore();
	do_rest();

So you have local irq disable/enable in both cases. So for the case where you don't have a local cmpxchg8b/16b available it's not worse versus irq disable/enable than now. It just has the possible repeat case when stuff changed between the prep and the actual cmpxchg, which is the same problem when cmpxchg8b/16 is available.

Thanks,

tglx

Christoph Lameter

2011-05-04 15:00 UTC #41 lore raw

Quoted 5 lines

So you have local irq disable/enable in both cases. So for the case where you don't have a local cmpxchg8b/16b available it's not worse versus irq disable/enable than now. It just has the possible repeat case when stuff changed between the prep and the actual cmpxchg, which is the same problem when cmpxchg8b/16 is available.

Right there is only the tid management that is added. Hope I am fast enough to at least get one patch in (not very well tested):

Subject: slub: Remove CONFIG_CMPXCHG_LOCAL ifdeffery

Remove the #ifdefs. This means that the irqsafe_cpu_cmpxchg_double() is used everywhere.

There may be performance implications since:

A. We now have to manage a transaction ID for all arches

B. The interrupt holdoff for arches not supporting CONFIG_CMPXCHG_LOCAL is reduced to a very short irqoff section.

There are no multiple irqoff/irqon sequences as a result of this change. Even in the fallback case we only have to do one disable and enable like before.

Signed-off-by Christoph Lameter <[email protected]>

---
 include/linux/slub_def.h |    2 -
 mm/slub.c                |   56 -----------------------------------------------
 2 files changed, 58 deletions(-)

include/linux/slub_def.h +0 -2

Index: linux-2.6/include/linux/slub_def.h
===================================================================
@@ -37,9 +37,7 @@ enum stat_item {
 
 struct kmem_cache_cpu {
 	void **freelist;	/* Pointer to next available object */
-#ifdef CONFIG_CMPXCHG_LOCAL
 	unsigned long tid;	/* Globally unique transaction id */
-#endif
 	struct page *page;	/* The slab from which we are allocating */
 	int node;		/* The node of the page (or -1 for debug) */
 #ifdef CONFIG_SLUB_STATS

mm/slub.c +0 ~~-56~~

203 lines not on this page · raw message

Linus Torvalds

2011-05-04 15:14 UTC #42 lore raw

Quoted 3 lines · Christoph Lameter

Right there is only the tid management that is added. Hope I am fast enough to at least get one patch in (not very well tested):

So the thing that worries me about this is non-x86 architectures.

Have we verified that the generic routines are ok for all architectures? Has somebody checked the memory barriers in particular? Things that work on x86 may not work on non-x86. Everything should be per-cpu _except_ for the initialization, I think, but that should be double-checked.

I guess the initialization happens so early that we don't really need to worry about it, but I'd still like somebody to really double- and triple-check it for me.

My gut reaction would be: let's do the minimal patch that just fixes things to do irqsafe_cpu_cmpxchg_double() for 2.6.39, and then let's remove the #ifdef'fery in -rc1. Or make _really_ sure that things are ok for platforms that never even triggered the CMPXCHG_LOCAL case before.

Hmm?

Linus

Ingo Molnar

2011-05-04 15:21 UTC #39 lore raw

Quoted 8 lines · Christoph Lameter

Quoted 5 lines · Tejun Heo
The cmpxchg_double optimization made the fast path go quite faster when CPU supports it but it may as well slow things down considerably if CPU doesn't, due to extra irq on/off's. Anyways, here's hoping that the slow down is acceptable compared to the base code without cmpxchg_double and the ugliness can be removed.

Quoted 8 lines

I think we are all in agreement. First path to fix the percpu macros:
Subject: percpu: Fix irqsafe_cpu_cmpxchg_double
The function in the macro was not updated when the function was given a bool return value.
Signed-off-by Christoph Lameter <[email protected]>

Note, the final commit needs to have a proper Reported-by and explanation about how this was triggered, to fairly credit all the hard work done by Werner ...

Thanks,

Ingo

Christoph Lameter

2011-05-04 15:28 UTC #43 lore raw

Quoted 9 lines · Linus Torvalds

Quoted 3 lines · Christoph Lameter
Right there is only the tid management that is added. Hope I am fast enough to at least get one patch in (not very well tested):
So the thing that worries me about this is non-x86 architectures.
Have we verified that the generic routines are ok for all architectures? Has somebody checked the memory barriers in particular?

There are no memory barriers used. The barrier() here is a compiler hint to not keep data across it. This is dealing with concurrency issues on the *same* cpu due to preemption and irqs. It avoids the interrupt disable. There no memory barrier issues for code running on a single cpu. The cpu is always guaranteed to have a consistent view of the data.

There is an additional patchset that also uses a cmpxchg_double for the slowpath. There barrier issues may arise because concurrent access is possible but that is likely to be merged only in 2.6.41. The cmpxchg_double in that case is a locked operation.

Quoted 3 lines

Things that work on x86 may not work on non-x86. Everything should be per-cpu _except_ for the initialization, I think, but that should be double-checked.

The fallback path is using interrupt disable / enable. This must be working on all arches AFAICT.

Quoted 7 lines

My gut reaction would be: let's do the minimal patch that just fixes things to do irqsafe_cpu_cmpxchg_double() for 2.6.39, and then let's remove the #ifdef'fery in -rc1. Or make _really_ sure that things are ok for platforms that never even triggered the CMPXCHG_LOCAL case before.
Hmm?

Ok. Pekka can put that patch in for the next round of merges?

Linus Torvalds

2011-05-04 15:38 UTC #44 lore raw

Quoted 3 lines · Christoph Lameter

There are no memory barriers used. The barrier() here is a compiler hint to not keep data across it.

Go back and read my email, please.

The lack of memory barriers is exactly what I worry about.

Look at the initialization path. The "tid" thing is _not_ purely cpu-local.

Linus

Pekka Enberg

2011-05-04 15:41 UTC #75 lore raw

Quoted 9 lines · Christoph Lameter

Subject: slub: Remove CONFIG_CMPXCHG_LOCAL ifdeffery
Remove the #ifdefs. This means that the irqsafe_cpu_cmpxchg_double() is used everywhere.
There may be performance implications since:
A. We now have to manage a transaction ID for all arches

Quoted 7 lines

B. The interrupt holdoff for arches not supporting CONFIG_CMPXCHG_LOCAL is reduced to a very short irqoff section.
There are no multiple irqoff/irqon sequences as a result of this change. Even in the fallback case we only have to do one disable and enable like before.
Signed-off-by Christoph Lameter <[email protected]>

This doesn't apply cleanly on top of slab/next which has some of your cleanup patches applied. There's some CONFIG_PREEMPT conflicts so I'd rather you rediffed it yourself.

Pekka

Pekka Enberg

2011-05-04 15:45 UTC #46 lore raw

Quoted 9 lines · Christoph Lameter

Quoted 7 lines
My gut reaction would be: let's do the minimal patch that just fixes things to do irqsafe_cpu_cmpxchg_double() for 2.6.39, and then let's remove the #ifdef'fery in -rc1. Or make _really_ sure that things are ok for platforms that never even triggered the CMPXCHG_LOCAL case before.
Hmm?
Ok. Pekka can put that patch in for the next round of merges?

Sure. Who's taking care of the minimal fix for .39?

Linus Torvalds

2011-05-04 15:59 UTC #47 lore raw

Quoted 2 lines · Pekka Enberg

Sure. Who's taking care of the minimal fix for .39?

I'll take it. I'm just hoping to also get Werner's tested-by for it.

I'm pretty confident this is it, though, so if I don't get it by the end of the day I'll just apply it regardless with just a reported-by from him.

Linus

Christoph Lameter

2011-05-04 16:08 UTC #45 lore raw

Quoted 10 lines · Linus Torvalds

Quoted 3 lines · Christoph Lameter
There are no memory barriers used. The barrier() here is a compiler hint to not keep data across it.
Go back and read my email, please.
The lack of memory barriers is exactly what I worry about.
Look at the initialization path. The "tid" thing is _not_ purely cpu-local.

kmem_cache_create() only hands the pointer to the slab cache back after the percpu values have been initialized. In order to do an allocation one needs to have the pointer to the slab cache.

You think that accesses to struct kmem_cache_cpu by other cpus could require memory barriers? But this is what both SLAB and SLUB already do today even before these modification. Nor do we have memory barriers for the same situation in the page allocator.

Ingo Molnar

2011-05-04 16:31 UTC #32 lore raw

Re: [PATCH] slub: Fix the lockless code on 32-bit platforms with no 64-bit cmpxchg

Quoted 9 lines · Ingo Molnar

Quoted 6 lines · Ingo Molnar
Quoted 1 line · Thomas Gleixner
Patch below. Ingo, can you test that please ?
Sure - test underway.

Quoted 7 lines

Ok, the patch below is looking really good - the test would have crashed on the bad kernel. I think we can consider the regression fixed:
Acked-and-tested-by Ingo Molnar <[email protected]>
I'll keep it running some more to make really sure it's fixed, plus it would be nice if Walter tested your fix as well.

Stopped the test after 2 hours uptime - i think we can consider the crash fixed.

Thanks,

Ingo

Ingo Molnar

2011-05-04 16:50 UTC #73 lore raw

Quoted 3 lines · Linus Torvalds

My gut reaction would be: let's do the minimal patch that just fixes things to do irqsafe_cpu_cmpxchg_double() for 2.6.39, and then let's remove the #ifdef'fery in -rc1. [...]

Looks like the sanest option IMHO, -rc7 is pretty late for anything than a few-liner patch.

Quoted 2 lines

[...] Or make _really_ sure that things are ok for platforms that never even triggered the CMPXCHG_LOCAL case before.

Considering that the status quo was !CMPXCHG_LOCAL in v2.6.38 and that lockless SLUB is an x86-only affair right now:

 $ git grep CMPXCHG_LOCAL arch/
 arch/um/Kconfig.x86:config CMPXCHG_LOCAL
 arch/x86/Kconfig.cpu:config CMPXCHG_LOCAL

There should be no problem with other architectures, right?

Thanks,

Ingo

Thomas Gleixner

2011-05-04 17:12 UTC #74 lore raw

Quoted 9 lines · Ingo Molnar

Quoted 3 lines · Linus Torvalds
My gut reaction would be: let's do the minimal patch that just fixes things to do irqsafe_cpu_cmpxchg_double() for 2.6.39, and then let's remove the #ifdef'fery in -rc1. [...]
Looks like the sanest option IMHO, -rc7 is pretty late for anything than a few-liner patch.

Agreed.

Quoted 11 lines

Quoted 2 lines
[...] Or make _really_ sure that things are ok for platforms that never even triggered the CMPXCHG_LOCAL case before.
Considering that the status quo was !CMPXCHG_LOCAL in v2.6.38 and that lockless SLUB is an x86-only affair right now:
 $ git grep CMPXCHG_LOCAL arch/
 arch/um/Kconfig.x86:config CMPXCHG_LOCAL
 arch/x86/Kconfig.cpu:config CMPXCHG_LOCAL
There should be no problem with other architectures, right?

No, they use the local_irq disabled path which is pretty much the same as in 38.

Thanks,

tglx

Linus Torvalds

2011-05-04 18:20 UTC #48 lore raw

Quoted 6 lines · Linus Torvalds

I'll take it. I'm just hoping to also get Werner's tested-by for it.
I'm pretty confident this is it, though, so if I don't get it by the end of the day I'll just apply it regardless with just a reported-by from him.

So I'm still waiting for the tested-by, but in the meantime I wrote a changelog. And part of that changelog reads:

[ Btw, that whole "generic code defaults to no protection" design just
sounds stupid - if the code needs no protection, there is no reason to use "cmpxchg_double" to begin with. So we should probably just remove
the unprotected version entirely as pointless. - Linus ]

which really sums up the whole thing.

The current "this_cpu_cmpxchg_double()" implementation is just incredibly idiotic. There is absolutely _no_ point to having that function at all. Why does it exist?

I can kind of see the point of the "preempt" version, although I'm not entirely convinced of that either. But the notion of having a "cmpxchg" function that isn't atomic even on a single CPU just makes me go "f*ck that, whoever wrote that is just a moron".

If the function doesn't need atomicity, you're really better off just writing it out. It's going to be faster on pretty much all architectures using just regular load/store instructions.

Linus

Christoph Lameter

2011-05-04 18:49 UTC #49 lore raw

Quoted 8 lines · Linus Torvalds

So I'm still waiting for the tested-by, but in the meantime I wrote a changelog. And part of that changelog reads:
[ Btw, that whole "generic code defaults to no protection" design just
sounds stupid - if the code needs no protection, there is no reason to use "cmpxchg_double" to begin with. So we should probably just remove
the unprotected version entirely as pointless. - Linus ]

The unprotected version is the __this_cpu_cmpxchg_double? That currently has no user and could be removed. But all other functions also have __ variants so it was put there for symmetries sake.

Quoted 5 lines

which really sums up the whole thing.
The current "this_cpu_cmpxchg_double()" implementation is just incredibly idiotic. There is absolutely _no_ point to having that function at all. Why does it exist?

this_cpu_cmpxchg_double() affords protection against preemption but not irqs. In the allocator case we have to deal with interrupts so its not useful there. Other code that can guarantee that nothing runs from irq context can use this function and then the fallback handling on other arches can avoid disabling and enabling interrupts which is required by irqsafe_cpu_cmpxchg_double().

Quoted 8 lines

I can kind of see the point of the "preempt" version, although I'm not entirely convinced of that either. But the notion of having a "cmpxchg" function that isn't atomic even on a single CPU just makes me go "f*ck that, whoever wrote that is just a moron".
If the function doesn't need atomicity, you're really better off just writing it out. It's going to be faster on pretty much all architectures using just regular load/store instructions.

The unlocked cmpxchg8b/16b is quite fast on x86. The __ function allows the use of cmpxchg8b on x86 and the fallback (and therefore longer) code on other archs.

But I see the point that __this_cpu_cmpxchg double is pretty useless. There is no user and so it could be removed.

Linus Torvalds

2011-05-04 19:08 UTC #50 lore raw

Quoted 4 lines · Christoph Lameter

The unprotected version is the __this_cpu_cmpxchg_double? That currently has no user and could be removed. But all other functions also have __ variants so it was put there for symmetries sake.

No, I'm talking about the regular "this_cpu_cmpxchg_double" with no underscores.

Why the f*!@ does that exist?

Why the f*%^ do you have to write "irqsafe", when the whole concept DOES NOT MAKE SENSE without the "irqsafe"?

Why did we have this bug in the first place, in other words? The whole interface was mis-designed, and pretty much designed to cause bugs.

I think we should remove every single version of "this_cpu_cmpxchg_double" except for two: the per-cpu one and the SMP-safe one.

And the per-cpu one doesn't mention "irqsafe" or "preempt" or anything like that, because the whole function makes no sense except when it is irq-safe and preempt-safe.

So I think the fact that we need to say "irqsafe" is a bug. Plain and simple.

The whole (and ONLY) point of "cmpxchg" is atomicity.

This is not like "add one to something". That's an operation that makes sense outside of atomicity issues. But "cmpxchg" is all about being atomic.

Linus

Christoph Lameter

2011-05-04 19:30 UTC #51 lore raw

Quoted 9 lines · Linus Torvalds

Quoted 4 lines · Christoph Lameter
The unprotected version is the __this_cpu_cmpxchg_double? That currently has no user and could be removed. But all other functions also have __ variants so it was put there for symmetries sake.
No, I'm talking about the regular "this_cpu_cmpxchg_double" with no underscores.
Why the f*!@ does that exist?

Because it is useful if the per cpu serialization only requires safety from preemption during the cmpxchg.

Quoted 2 lines

Why the f*%^ do you have to write "irqsafe", when the whole concept DOES NOT MAKE SENSE without the "irqsafe"?

Because that is how the other irq safe this_cpu_xxx functions are named and I tried to keep it consistent.

Quoted 3 lines

I think we should remove every single version of "this_cpu_cmpxchg_double" except for two: the per-cpu one and the SMP-safe one.

The smp safe one would be cmpxchg_double() then. This is part of a future patchset and would be named like other fully atomic ops. Would not be part of include/linux/percpu.h because it is not a per cpu related function.

Quoted 3 lines

And the per-cpu one doesn't mention "irqsafe" or "preempt" or anything like that, because the whole function makes no sense except when it is irq-safe and preempt-safe.

It does make sense because arches that do not have the hardware capabilities must use fallback implementations that can be faster if f.e. irqs must not be disabled. And this_cpu_xxx ops are usually used in performance critical paths.

Quoted 7 lines

So I think the fact that we need to say "irqsafe" is a bug. Plain and simple.
The whole (and ONLY) point of "cmpxchg" is atomicity.
This is not like "add one to something". That's an operation that makes sense outside of atomicity issues. But "cmpxchg" is all about being atomic.

The naming convention came about from the existing this_cpu_xxx operations (and yes those starting with the problem of the increment). To do it differently now just for this function would make it more difficult to comprehend for someone already familiar with this_cpu operations on per cpu data.

Linus Torvalds

2011-05-04 19:45 UTC #52 lore raw

Quoted 3 lines · Christoph Lameter

The naming convention came about from the existing this_cpu_xxx operations

You're missing my point.

An "add" operation makes sense even if it isn't atomic, because atomicity isn't a part of the definition of "add".

But cmpxchg DOES NOT MAKE SENSE without atomicity guarantees.

The whole operation is about atomicity.

Having a version that isn't atomic is STUPID. It's misleading. It's _wrong_.

In contrast, having a non-atomic "add" version is understandable.

So when you say "naming convention", you're missing the much bigger naming convention. Namely the "cmpxchg" part!

Linus

Christoph Lameter

2011-05-04 20:04 UTC #53 lore raw

Quoted 11 lines · Linus Torvalds

Quoted 3 lines · Christoph Lameter
The naming convention came about from the existing this_cpu_xxx operations
You're missing my point.
An "add" operation makes sense even if it isn't atomic, because atomicity isn't a part of the definition of "add".
But cmpxchg DOES NOT MAKE SENSE without atomicity guarantees.

This is not a real cmpxchg after all. Its not atomic in the sense of other functions. Its only "percpu atomic" if you want it that way. This is *not* a full cmpxchg_double().

Quoted 3 lines

The whole operation is about atomicity.
Having a version that isn't atomic is STUPID. It's misleading. It's _wrong_.

Its "atomic" in the sense that it is an instruction that is either executed or not in total and that fact alone allow the avoiding of synchronization for preemption and interrupts. We just push as much processing as possible into this single instruction and then we dont have to worry about preemption or interrupts while this function is executed by the processor.

Quoted 4 lines

In contrast, having a non-atomic "add" version is understandable.
So when you say "naming convention", you're missing the much bigger naming convention. Namely the "cmpxchg" part!

Well this is not really a true cmpxchg. There is no lock prefix.

The semantics of the this_cpu_xxx functions are not atomic but only per cpu atomic. That per cpu atomicity can require only the exclusion of preemption or the exclusion of interrupts.

In extreme cases we dont care about preemption or interrupts interfering with the operation. We just want to opportunistically take advantage of sophisticated instructions if they are available (f.e. for accurate vm counters). Or we may have some other external means of serialization (like lock or we already disabled preemption). Thats what the __ operations are for.

Maybe I should have pushed the cmpxchg_double() before the this_cpu_cmpxchg to avoid these misunderstandings

Here is the patch for the fullly atomic cmpxchg_double() which will be needed for making the non per cpu specific processing lockless later:

Subject: x86: Add support for cmpxchg_double

A simple implementation that only supports the word size and does not have a fallback mode (would require a spinlock).

And 32 and 64 bit support for cmpxchg_double. cmpxchg double uses the cmpxchg8b or cmpxchg16b instruction on x86 processors to compare and swap 2 machine words. This allows lockless algorithms to move more context information through critical sections.

Set a flag CONFIG_CMPXCHG_DOUBLE to signal the support of that feature during kernel builds.

Signed-off-by Christoph Lameter <[email protected]>

---
 arch/x86/Kconfig.cpu              |    3 ++
 arch/x86/include/asm/cmpxchg_32.h |   46 ++++++++++++++++++++++++++++++++++++++
 arch/x86/include/asm/cmpxchg_64.h |   45 +++++++++++++++++++++++++++++++++++++
 arch/x86/include/asm/cpufeature.h |    1
 4 files changed, 95 insertions(+)

arch/x86/include/asm/cmpxchg_64.h +45 -0

Index: linux-2.6/arch/x86/include/asm/cmpxchg_64.h
===================================================================
@@ -151,4 +151,49 @@ extern void __cmpxchg_wrong_size(void);
 	cmpxchg_local((ptr), (o), (n));					\
 })
 
+#define cmpxchg16b(ptr, o1, o2, n1, n2)				\
+({								\
+	char __ret;						\
+	__typeof__(o2) __junk;					\
+	__typeof__(*(ptr)) __old1 = (o1);			\
+	__typeof__(o2) __old2 = (o2);				\
+	__typeof__(*(ptr)) __new1 = (n1);			\
+	__typeof__(o2) __new2 = (n2);				\
+	asm volatile(LOCK_PREFIX_HERE "lock; cmpxchg16b (%%rsi);setz %1" \
+		       : "=d"(__junk), "=a"(__ret)		\
+		       : "S"(ptr), "b"(__new1),	"c"(__new2),	\
+		         "a"(__old1), "d"(__old2));		\
+	__ret; })
+
+
+#define cmpxchg16b_local(ptr, o1, o2, n1, n2)			\
+({								\
+	char __ret;						\
+	__typeof__(o2) __junk;					\
+	__typeof__(*(ptr)) __old1 = (o1);			\
+	__typeof__(o2) __old2 = (o2);				\
+	__typeof__(*(ptr)) __new1 = (n1);			\
+	__typeof__(o2) __new2 = (n2);				\
+	asm volatile("cmpxchg16b (%%rsi)\n\t\tsetz %1\n\t"	\
+		       : "=d"(__junk)_, "=a"(__ret)		\
+		       : "S"((ptr)), "b"(__new1), "c"(__new2),	\
+ 		         "a"(__old1), "d"(__old2));		\
+	__ret; })
+
+#define cmpxchg_double(ptr, o1, o2, n1, n2)				\
+({									\
+	BUILD_BUG_ON(sizeof(*(ptr)) != 8);				\
+	VM_BUG_ON((unsigned long)(ptr) % 16);				\
+	cmpxchg16b((ptr), (o1), (o2), (n1), (n2));			\
+})
+
+#define cmpxchg_double_local(ptr, o1, o2, n1, n2)			\
+({									\
+	BUILD_BUG_ON(sizeof(*(ptr)) != 8);				\
+	VM_BUG_ON((unsigned long)(ptr) % 16);				\
+	cmpxchg16b_local((ptr), (o1), (o2), (n1), (n2));		\
+})
+
+#define system_has_cmpxchg_double() cpu_has_cx16
+
 #endif /* _ASM_X86_CMPXCHG_64_H */

arch/x86/include/asm/cmpxchg_32.h +46 -0

50 lines not on this page · raw message

arch/x86/Kconfig.cpu +3 -0

Index: linux-2.6/arch/x86/Kconfig.cpu
===================================================================
@@ -308,6 +308,9 @@ config X86_CMPXCHG
 config CMPXCHG_LOCAL
 	def_bool X86_64 || (X86_32 && !M386)
 
+config CMPXCHG_DOUBLE
+	def_bool X86_64 || (X86_32 && !M386)
+
 config X86_L1_CACHE_SHIFT
 	int
 	default "7" if MPENTIUM4 || MPSC

arch/x86/include/asm/cpufeature.h +1 -0

Index: linux-2.6/arch/x86/include/asm/cpufeature.h
===================================================================
@@ -286,6 +286,7 @@ extern const char * const x86_power_flag
 #define cpu_has_hypervisor	boot_cpu_has(X86_FEATURE_HYPERVISOR)
 #define cpu_has_pclmulqdq	boot_cpu_has(X86_FEATURE_PCLMULQDQ)
 #define cpu_has_perfctr_core	boot_cpu_has(X86_FEATURE_PERFCTR_CORE)
+#define cpu_has_cx16		boot_cpu_has(X86_FEATURE_CX16)
 
 #if defined(CONFIG_X86_INVLPG) || defined(CONFIG_X86_64)
 # define cpu_has_invlpg		1

[email protected]

2011-05-04 20:24 UTC #54 lore raw

Quoted 7 lines · Christoph Lameter

Quoted 1 line · Linus Torvalds
But cmpxchg DOES NOT MAKE SENSE without atomicity guarantees.
This is not a real cmpxchg after all. Its not atomic in the sense of other functions. Its only "percpu atomic" if you want it that way. This is *not* a full cmpxchg_double().

Calling it a cmpxchg when it doesn't have the primary distinguishing property of a hardware cmpxchg is just loading a bullet in the chamber and inviting kernel hackers to point it at their feet...

Christoph Lameter

2011-05-04 20:33 UTC #55 lore raw

Quoted 5 lines · On Wed, 4 May 2011, [email protected] wrote:

Quoted 4 lines · Christoph Lameter
Quoted 1 line · Linus Torvalds
But cmpxchg DOES NOT MAKE SENSE without atomicity guarantees.

Quoted 7 lines

Quoted 3 lines
This is not a real cmpxchg after all. Its not atomic in the sense of other functions. Its only "percpu atomic" if you want it that way. This is *not* a full cmpxchg_double().
Calling it a cmpxchg when it doesn't have the primary distinguishing property of a hardware cmpxchg is just loading a bullet in the chamber and inviting kernel hackers to point it at their feet...

It does have most of the distinguishing characterstics but the lock-prefixless cmpxchg8b/16b (which is quite fast) is used in a unique way here in a percpu fastdpath. Thats why we have the strange naming this_cpu_cmpxchg_double etc.

Ingo Molnar

2011-05-04 20:49 UTC #56 lore raw

Quoted 6 lines · Christoph Lameter

Quoted 3 lines
Calling it a cmpxchg when it doesn't have the primary distinguishing property of a hardware cmpxchg is just loading a bullet in the chamber and inviting kernel hackers to point it at their feet...
It does have most of the distinguishing characterstics but the lock-prefixless cmpxchg8b/16b (which is quite fast) [...]

6 cycles for CMPXCHG8B versus 19 cycles for LOCK CMPXCHG8B, on Nehalem.

And note that it's not really true that the LOCK prefix-less CMPXCHG8B is not
atomic: the write is atomic if the target word is naturally aligned, on i586 and upwards ...

So in practice, unless the SLUB variables are misaligned, the lock prefix-less CMPXCHG8B *IS* atomic.

Non-atomicity is a special case of a weird special case.

Quoted 2 lines

[...] is used in a unique way here in a percpu fastdpath. Thats why we have the strange naming this_cpu_cmpxchg_double etc.

Furthermore, we never used cmpxchg in the kernel without expecting atomicity.

Uniquely strange, unintuitive naming == invitation for strange bugs.

And guess what, we had a strange bug here. Can you possibly see any connection?

Thanks,

Ingo

Linus Torvalds

2011-05-04 21:07 UTC #57 lore raw

Quoted 3 lines · Christoph Lameter

Maybe I should have pushed the cmpxchg_double() before the this_cpu_cmpxchg to avoid these misunderstandings

Christoph, the only mis-understanding is yours.

I understand perfectly that the percpu cmpxchg isn't SMP-atomic. We all know that.

The problem is that cmpxchg *IS*NOT*AN*CMPXCHG*AT*ALL*. Not even on UP.

It's something totally different.

Linus

Linus Torvalds

2011-05-04 21:20 UTC #58 lore raw

Quoted 2 lines · Linus Torvalds

The problem is that cmpxchg *IS*NOT*AN*CMPXCHG*AT*ALL*. Not even on UP.

Btw, the really sad thing is that as far as I can tell, the cmpxchg64_local() function that we define actually gets this *right*, with a helper function that actually works (and disables interrupts) and all the alternative_io() stuff to then use the right instruction automatically if the CPU supports it, rather than making it a hardcoded decision.

But we don't use it, because the "percpu_cmpxchg8b_double()" code rolls its own inferior version, and has slightly different semantics (ie the whole "return value success" thing).

So close, but yet so far.

Linus

Thomas Gleixner

2011-05-04 21:41 UTC #59 lore raw

Quoted 12 lines · Linus Torvalds

Quoted 2 lines · Linus Torvalds
The problem is that cmpxchg *IS*NOT*AN*CMPXCHG*AT*ALL*. Not even on UP.
Btw, the really sad thing is that as far as I can tell, the cmpxchg64_local() function that we define actually gets this *right*, with a helper function that actually works (and disables interrupts) and all the alternative_io() stuff to then use the right instruction automatically if the CPU supports it, rather than making it a hardcoded decision.

Quoted 3 lines

But we don't use it, because the "percpu_cmpxchg8b_double()" code rolls its own inferior version, and has slightly different semantics (ie the whole "return value success" thing).

Yeah, I tripped over that as well. And the new shiny extra CONFIG_CMPXCHG_DOUBLE misfeature Christoph nailed together is just ignoring it as well. It's just more useless #ifdef and CONFIG bloat.

That whole this_cpu_* stuff seems to be designed by committee. A quick grep shows that max. 10% of the implemented macro hell is actually used. Can we get rid of all unused ones and bring them back when they are actually required?

Thanks,

tglx

Ben Greear

2011-05-04 21:56 UTC #33 lore raw

Re: [PATCH] slub: Fix the lockless code on 32-bit platforms with no 64-bit cmpxchg

In reply to a message not in the archive

Quoted 4 lines · Ingo Molnar

Quoted 1 line
From c22bd309573330e33a77c329405d9473fc14f1cb Mon Sep 17 00:00:00 2001
From: Thomas Gleixner<[email protected]>
Date: Wed, 4 May 2011 15:38:19 +0200
Subject: [PATCH] slub: Fix the lockless code on 32-bit platforms with no 64-bit cmpxchg

This patch appears to fix the crashes I was seeing on my 32-bit Atom system.

It would crash within 30 seconds of booting every time before this patch...

Thanks,
Ben

Signature · 2 lines

Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com

Linus Torvalds

2011-05-04 22:01 UTC #34 lore raw

Re: [PATCH] slub: Fix the lockless code on 32-bit platforms with no 64-bit cmpxchg

Quoted 5 lines · Ben Greear

This patch appears to fix the crashes I was seeing on my 32-bit Atom system.
It would crash within 30 seconds of booting every time before this patch...

Oh, well - I already committed it and pushed out, otherwise I'd have added that piece of information to the commit log.

But it's good to know that others had seen this too, just never realized what was going on.

Btw, that does seem to imply that your kernel config is somewhat odd. We _should_ be using cmpxchg8b natively if you compile for anything newer than PPro, and that includes atom.

Did you perhaps say "compile for Pentium" (which is pretty close to Atom in some respects - but we don't trust that all Pentium-class CPU's have cmpxchg8b, even if the Intel ones all should).

Regardless, it's pushed out and should be in current -git, apart from any possible mirroring delays there may be that may b eholdin git up from actually being visible on the public kernel.org machines yet.

Linus

Ben Greear

2011-05-04 22:25 UTC #35 lore raw

Re: [PATCH] slub: Fix the lockless code on 32-bit platforms with no 64-bit cmpxchg

Quoted 9 lines · Linus Torvalds

Quoted 5 lines · Ben Greear
This patch appears to fix the crashes I was seeing on my 32-bit Atom system.
It would crash within 30 seconds of booting every time before this patch...
Oh, well - I already committed it and pushed out, otherwise I'd have added that piece of information to the commit log.

No worries...I'm just happy to finally be able to boot .39 :)

Quoted 10 lines

But it's good to know that others had seen this too, just never realized what was going on.
Btw, that does seem to imply that your kernel config is somewhat odd. We _should_ be using cmpxchg8b natively if you compile for anything newer than PPro, and that includes atom.
Did you perhaps say "compile for Pentium" (which is pretty close to Atom in some respects - but we don't trust that all Pentium-class CPU's have cmpxchg8b, even if the Intel ones all should).

Well, yes. I'm compiling for 'M586' it seems, plus SMP, pre-empt, etc.

Maybe it's time to move to a newer processor family!

Thanks,
Ben

Signature · 2 lines

Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com

werner

author

2011-05-05 06:46 UTC #72 lore raw

Just now I compiled 2.6.39-rc6-git1 (including still
the patchs by hand because not yet included), its also OK, good running, without all these problems. :)

Yes, I continue with my everythingyesconfig, which all the time and also now
after this patchs gives good generic kernels for the distro, working good on my
server and on the laptops of the neighbours except that sometimes
(instead of ELAN) I'll make 486 binaries. And now put the package on the mirrors because everything is OK, and I dont need to switch on and off the server so that they can sync, and the users can use the now working packages of 2.6.39-rcX , which have many new drivers and other good improvements.

Like since years, I continue to compile new versions since -rc1 and report problems. But it was very important that Linus take care of these problems now, because nobody else searched for the reasons. These bugs in logfs and in slub.c and percpu.h probably were processor-type / arch independent, and would have given problems for the most users, so that they could not have been ignored.
wl

=======================================================

Quoted 5 lines · On Thu, 5 May 2011 08:02:04 +0200 Ingo Molnar <[email protected]> wrote:

Quoted 1 line · werner
I'll test the patchs with a 486 config, switching off ELAN.

Quoted 9 lines

I think you can skip all those other test suggestions in this thread - as you confirmed in this email the root cause of your crashes has been found.
So the best testing you could do would be for you to resume testing whatever original config you were using on your system, and make sure there's really no crashes left - and report any other regressions you might see.
Thanks,
Ingo

"werner" <[email protected]>
---
Professional hosting for everyone - http://www.host.ru

Tejun Heo

maintainer

2011-05-05 09:54 UTC #60 lore raw

Hey,

Quoted 8 lines · Thomas Gleixner

Yeah, I tripped over that as well. And the new shiny extra CONFIG_CMPXCHG_DOUBLE misfeature Christoph nailed together is just ignoring it as well. It's just more useless #ifdef and CONFIG bloat.
That whole this_cpu_* stuff seems to be designed by committee. A quick grep shows that max. 10% of the implemented macro hell is actually used. Can we get rid of all unused ones and bring them back when they are actually required?

What happened there was more of lack of design rather than too much design. At first it were a few ops with only two variants - preemption protected default one and __ prefixed unprotected one. Then came the irqsafe ops and then the whole kinda exploded on itself with all the different ops.

Cleaning up percpu accessors has been on my todo list for some time now. The this_cpu_*() thing is still in its infancy and there aren't many users and the existing ones are mostly in the core, so cleaning things up should be relatively easy. Things which I've been considering are...

1. Static and dynamic accessors.

We have two sets of percpu accessors. Originally, one set was for static percpu variables and the other for dynamic ones. Since the percpu allocator reimplementation, there's no distinction between static and dynamic and the new this_cpu_*() functions don't distinguish them. We need to unify the duplicate ones. There's nothing much to decide about this. It just needs to be done.

2. this_cpu_*()

This one is more tricky. General cleanup aside...

this_cpu_*() ops currently come in three flavors and the naming convention is rather confusing. __this_cpu for no protection, this_cpu for preemtion disabled and irqsafe_cpu for irq protected. IIUC it was intended to loosely match the naming convention of spinlocks but I'm not sure whether that's beneficial in this case.

The biggest problem, as shown by this bug, is that it's error-prone. Percpu ops as they currently stand don't have lockdep protection for accessing contexts. There's nothing protecting percpu vars being accessed from wrong contexts. I think adding lockdep protection should be doable and should be done, but it would be much better if it's safer by default.

The problem is aggravated by the fact that, on x86 where most of developement and testing happens, there's no difference between __this_cpu_*(), this_cpu_*() and irqsafe_cpu_*(). They're one and the same, so testing coverage suffers. We can avoid this by adding a debug option which forces the generic versions whether the arch ones are there or not, but it does raise the question - do we really need all these different confusing variants?

For x86, it doesn't matter. There are some corner cases with cmpxchg_[double] but I don't think we would be in any trouble just using the generic ones for them. The problem is with other archs where local atomic ops haven't been or, for most load-store architectures, can't be implemented. We might say "eh... screw them" and let them use the generic ones but the problem is that this_cpu_*() tend to be used in extremely hot paths where extra irq on/off's can show up as significant overhead.

The thing that I want gone the most is the irqsafe_ variant. It would be much nicer/safer if this_cpu_*() is atomic against everything. If the caller is gonna take care of exclusion from the enclosing area (this is a valid use case when there's a series of operations to be done including but not limited to multiple this_cpu ops), it can use __this_cpu_*() ones.

The reason to disable preemtion instead of IRQ is two-fold - First, preemption can be disabled for longer period than IRQ. Second, depending on CPU, toggling preemption is significantly cheaper than toggling preemption. For this_cpu ops, the first one doesn't apply. It's always very short, so the only thing that needs to be considered is the overhead of toggling protection.

So, to avoid suffering performance penalty from this_cpu_*() conversion, architectures can choose one of the followings.

1. Implement arch specific this_cpu_*() ops. This would be the better way but unfortunately many architectures simply can't. :-(

2. Make irq toggling as cheap as preemption toggling. This can be achieved by implementing IRQ masking in software. I played with it a bit on x86 last year. I didn't get to finish it and it would have taken me some time to iron out weird failures but it didn't seem too difficult and as irq on/off is quite expensive on a lot of CPUs, this might bring overall performance benefit.

For many archs, #2 would be the only choice and if we're gonna do that I think it would be best to do it on x86 too. It involves changes to common code too and x86 has the highest test/development coverage.

I don't feel brave enough to remove preemption protected ones without first somehow taking care of non-x86 archs. The preemption disabled ones are used for statistics in net, block, irq and fs and simply switching to irq protected ones is likely to noticeably hurt many non-x86 archs. Maybe the ones which really matter can implement at least _add() and we can get away without doing soft irq masking.

Anyways, that's what I've been thinking. I'll get to it in the next devel cycle or the one after that. What do you guys think about soft irq masking idea?

Thanks.

--
tejun

Ingo Molnar

2011-05-05 10:18 UTC #61 lore raw

Quoted 10 lines · Tejun Heo

2. Make irq toggling as cheap as preemption toggling. This can be achieved by implementing IRQ masking in software. I played with it a bit on x86 last year. I didn't get to finish it and it would have taken me some time to iron out weird failures but it didn't seem too difficult and as irq on/off is quite expensive on a lot of CPUs, this might bring overall performance benefit.
For many archs, #2 would be the only choice and if we're gonna do that I think it would be best to do it on x86 too. It involves changes to common code too and x86 has the highest test/development coverage.

We played with this in -rt on and off but note that -rt doesnt do this right now. Interestingly, most of the irq-disable wrappery and state tracking code for that is upstream already, via the lockdep irq state tracking patches. (Surprise! :-)

The disadvantages:

- register pressure increases, the pushf+cli+popf sequence has no register side-effects, while soft flags inevitably disturb register allocations. *possibly* quite low with the modern percpu implementation, but this has to be measured very carefuly, with disassembly.

- icache size increases - the percpu ops are larger than the minimal pushf+cli+popfl sequence. Again, this too has to be measured both via vmlinux size analysis and via perf stat --repeat icache pressure runs.

[ This is also an assymetric cost: it increases the cost of the cache-cold
case, while most of the benefits are in the cache-hot case. ]

- irq replay becomes common and there's extra cost due to that. Also we are not ready to replay some types of irqs (lapic timer), at least with current code. So there's some ongoing maintenance cost there.

The benefits are:

- lockdep is already tracking irqs on/off sections rather carefully, so we know all the places that play with irqs and the ongoing maintenance cost is shared with what we'd have to do with lockdep anyway.

- on Nehalem a "PUSHF; CLI; POPF" sequence is 18 cycles, a soft sequence would be more like 2 cycles. So we win around 15 cycles per sequence in the fast path - minus collateral slowpath cost above ... which are not directly comparable.

- Stock mainline would become a truly hard RT irq handlers kernel which never ever disables hardirqs. Big wow factor and precision guided, laser mounted sharks!

I probably missed a few factors, but these are the main concerns.

My firm judgement: "Dunno".

Thanks,

Ingo

Thomas Gleixner

2011-05-05 10:45 UTC #62 lore raw

Quoted 11 lines · Tejun Heo

2. Make irq toggling as cheap as preemption toggling. This can be achieved by implementing IRQ masking in software. I played with it a bit on x86 last year. I didn't get to finish it and it would have taken me some time to iron out weird failures but it didn't seem too difficult and as irq on/off is quite expensive on a lot of CPUs, this might bring overall performance benefit.
For many archs, #2 would be the only choice and if we're gonna do that I think it would be best to do it on x86 too. It involves changes to common code too and x86 has the highest test/development coverage.

Quoted 6 lines

I don't feel brave enough to remove preemption protected ones without first somehow taking care of non-x86 archs. The preemption disabled ones are used for statistics in net, block, irq and fs and simply switching to irq protected ones is likely to noticeably hurt many non-x86 archs. Maybe the ones which really matter can implement at least _add() and we can get away without doing soft irq masking.

There it's fine when we have certain use cases which just need preemption protection, but the function name should be clear about it and we should only have the functions which are actually required instead of every possible flavour of _OP.

Quoted 3 lines

Anyways, that's what I've been thinking. I'll get to it in the next devel cycle or the one after that. What do you guys think about soft irq masking idea?

It's doable, we played around with that in rt already and put it aside again :)

You need to do the check at the vector entry and then reinject the vector on local_irq_enable/restore(), so that's mostly hackery in the ASM entry code which includes to fixup the pushed flags so the reti wont reeneable interrupts.

That's halfways easy to do on x86, not sure about other architectures, and you have to carefully weigh the actual benefit against the overhead in the slow path case plus the inevitable hackery you need in each architecture implementation.

No strong opinion, but definitely worthwhile to toy with it :)

Thanks,

tglx

Christoph Lameter

2011-05-05 18:20 UTC #63 lore raw

Quoted 3 lines · Tejun Heo

Anyways, that's what I've been thinking. I'll get to it in the next devel cycle or the one after that. What do you guys think about soft irq masking idea?

Great idea. Would make the whole irq on/off business much cheaper.

Ingo Molnar

2011-05-05 19:14 UTC #64 lore raw

Quoted 7 lines · Christoph Lameter

Quoted 3 lines · Tejun Heo
Anyways, that's what I've been thinking. I'll get to it in the next devel cycle or the one after that. What do you guys think about soft irq masking idea?
Great idea. Would make the whole irq on/off business much cheaper.

The tradeoffs are *not at all* clear and the result (on x86) is not 'much cheaper', at all ...

In particular the irq-enable path gets complicated by the need to check the flag and call a hardirq handling function in that case - a far cry from the single-byte POPF instruction. It will be somewhat cheaper cycle-wise - but the code gets bloated, so the instruction cache impact has to be measured carefully. (See my other mail for details.)

There's also the fact that PUSHF+CLI+POPF sequence has been getting cheaper all the time with newer hardware generations. CLI+STI is even cheaper, 10 cycles both on Intel and AMD CPUs. So it's an optimization that might get narrower and narrower with every CPU generation.

Thanks,

Ingo

werner

author

2011-05-05 19:53 UTC #65 lore raw