Skip to contentSkip to conversation

hanging modprobe aes_s390

DISCUSSION Jan Glauber 3 participants 7 messages Feb 24, 2009 – Feb 28, 2009

Subsystems (3)
Crypto API s390 s390 crypto modules, prng driver, arch random
lore
Share & tools
lore

Jan Glauber

author
#0 lore raw

Hi Herbert,

commit 73d3864a4823abda19ebc4387b6ddcbf416e3a77 leads to a hanging modprobe aes_s390 process. If the process is interrupted the following oops occurs:

------------[ cut here ]------------
Badness at crypto/algapi.c:293
Modules linked in: aes_generic aes_s390(+) binfmt_misc
CPU: 0 Not tainted 2.6.27-rc5-00010-g1aa4ecd #23
Process modprobe (pid: 3401, task: 000000003ea2c048, ksp: 000000003eb13b78)
Krnl PSW : 0704100180000000 000000000013af2c (crypto_wait_for_test+0x74/0x80)
           R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:0 CC:1 PM:0 EA:3
Krnl GPRS: 00000000010a4000 000000003eb14000 fffffffffffffe00 0000000000000000
           000000000013af12 00000000003e8900 0000000000007aa8 0000000000000000
           0000000000458048 000000003eb10000 000003e000017000 000003e000000000
           000000003d6cc600 00000000002dcfc8 000000000013af12 000000003eb13d18
Krnl Code: 000000000013af1c: ebcff0a00004       lmg     %r12,%r15,160(%r15)
           000000000013af22: c0f4ffffedf1       brcl    15,138b04
           000000000013af28: a7f40001           brc     15,13af2a
          >000000000013af2c: a7f4fff6           brc     15,13af18
           000000000013af30: a7f40001           brc     15,13af32
           000000000013af34: a7f4fff2           brc     15,13af18
           000000000013af38: ebcff0780024       stmg    %r12,%r15,120(%r15)
           000000000013af3e: a7f13f00           tmll    %r15,16128
Call Trace:
([<000000000013af12>] crypto_wait_for_test+0x5a/0x80)
 [<000000000013b288>] crypto_register_alg+0x80/0x98
 [<000003e000017136>] aes_s390_init+0x136/0x1e4 [aes_s390]
 [<00000000000120c0>] do_one_initcall+0x40/0x17c
 [<000000000006618e>] sys_init_module+0xc6/0x1e4
 [<0000000000023d18>] sysc_noemu+0x10/0x16
 [<000002000011cda2>] 0x2000011cda2

Last Breaking-Event-Address:
[<000000000013af28>] crypto_wait_for_test+0x70/0x80

That only happens if the aes_generic module isn't loaded. If aes_generic is already present the aes_s390 loads without problems.

Any idea how to solve this? Is something missing in the fallback code that uses aes_generic?

Cheers, Jan

maintainer
#1 lore raw
Quoted 6 lines · Jan Glauber

That only happens if the aes_generic module isn't loaded. If aes_generic is already present the aes_s390 loads without problems.

Any idea how to solve this? Is something missing in the fallback code that uses aes_generic?

Yeah it looks like it's waiting for the fallback to come up. However, the interesting bit is in the other processes, i.e., the one that's actually testing aes_s390 or constructing the aes-generic.

Could you get those back traces please?

Thanks,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
author
#2 lore raw
Quoted 5 lines · Herbert Xu
Quoted 4 lines · Jan Glauber

That only happens if the aes_generic module isn't loaded. If aes_generic is already present the aes_s390 loads without problems.

Quoted 8 lines
Quoted 2 lines

Any idea how to solve this? Is something missing in the fallback code that uses aes_generic?

Yeah it looks like it's waiting for the fallback to come up. However, the interesting bit is in the other processes, i.e., the one that's actually testing aes_s390 or constructing the aes-generic.

Could you get those back traces please?

I've found 4 processes/threads that seem to be involved. And in case that is relevant the oops message occured 3 times, see the attachment.

STACK TRACE FOR TASK: 0x394e37c8 (modprobe)

STACK:
0 schedule+1136 [0x2df82c]
1 schedule_timeout+198 [0x2e016e]
2 wait_for_common+256 [0x2df14c]
3 wait_for_completion_interruptible+56 [0x2df2ac] 4 crypto_wait_for_test+94 [0x16a28a]
5 crypto_register_alg+132 [0x16a598]
6 <sym not found>+<ERROR> [0x3e000daa13a] 7 do_one_initcall+68 [0x120c4]
8 sys_init_module+206 [0x825da]
9 sysc_noemu+16 [0x27a5e]

STACK TRACE FOR TASK: 0x3dcc4a60 (cryptomgr_test)

STACK:
0 schedule+1136 [0x2df82c]
1 schedule_timeout+198 [0x2e016e]
2 wait_for_common+256 [0x2df14c]
3 wait_for_completion+56 [0x2df344]
4 call_usermodehelper_exec+152 [0x5aec8]
5 request_module+272 [0x5b15c]
6 crypto_larval_lookup+176 [0x1681ec]
7 crypto_alg_mod_lookup+70 [0x1682f6]
8 crypto_alloc_base+70 [0x168416]
9 <sym not found>+<ERROR> [0x3e000d92d18]
10 __crypto_alloc_tfm+166 [0x16797a]
11 crypto_alloc_base+100 [0x168434]
12 alg_test+280 [0x17159c]
13 cryptomgr_test+80 [0x16fb4c]
14 kthread+106 [0x60d4a]
15 kernel_thread_starter+6 [0x1b01a]

STACK TRACE FOR TASK: 0x394fb7c8 (khelper)

STACK:
0 schedule+1136 [0x2df82c]
1 do_wait+672 [0x480c0]
2 sys_wait4+160 [0x48330]
3 wait_for_helper+102 [0x5afe2]
4 kernel_thread_starter+6 [0x1b01a]

STACK TRACE FOR TASK: 0x3f6137c8 (modprobe)

STACK:
0 schedule+1136 [0x2df82c]
1 fcntl_setlk+412 [0x107a0c]
2 sys_fcntl+262 [0xd7076]
3 sysc_noemu+16 [0x27a5e]

Cheers, Jan

aes_s390.cb83bb: AES hardware acceleration is only available for 128-bit keys

------------[ cut here ]------------
Badness at crypto/algapi.c:293
Modules linked in: aes_generic aes_s390(+) qeth_l3 qeth_l2 binfmt_misc zfcp qeth ccwgroup
CPU: 5 Not tainted 2.6.28 #25
Process modprobe (pid: 5270, task: 000000003c1e5cf8, ksp: 000000003be83a18)
Krnl PSW : 0704100180000000 000000000016a2ac (crypto_wait_for_test+0x80/0x8c)
           R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:0 CC:1 PM:0 EA:3
Krnl GPRS: 0000000000000000 000000003be84000 fffffffffffffe00 0000000000eb8e6e
           000000000016a28a 00000000002f1b60 00000000000476b1 0000000000000000
           0000000000673048 000000003be80000 000003e000000000 000000003be83bf8
           000000003d866400 00000000002ffd20 000000000016a28a 000000003be83bf8
Krnl Code: 000000000016a2a0: ebbfb0a00004	lmg	%r11,%r15,160(%r11)
           000000000016a2a6: 07f4		bcr	15,%r4
           000000000016a2a8: a7f40001		brc	15,16a2aa
          >000000000016a2ac: a7f4fff2		brc	15,16a290
           000000000016a2b0: a7f40001		brc	15,16a2b2
           000000000016a2b4: a7f4ffee		brc	15,16a290
           000000000016a2b8: ebbff0700024	stmg	%r11,%r15,112(%r15)
           000000000016a2be: a7f13f00		tmll	%r15,16128
Call Trace:
([<000000000016a28a>] crypto_wait_for_test+0x5e/0x8c)
 [<000000000016a598>] crypto_register_alg+0x84/0x9c
 [<000003e000bce13a>] aes_s390_init+0x13a/0x1f0 [aes_s390]
 [<00000000000120c4>] do_one_initcall+0x44/0x184
 [<00000000000825da>] sys_init_module+0xce/0x1f4
 [<0000000000027a5e>] sysc_noemu+0x10/0x16
 [<000002000011fda2>] 0x2000011fda2
no locks held by modprobe/5270.
Last Breaking-Event-Address:
 [<000000000016a2a8>] crypto_wait_for_test+0x7c/0x8c
------------[ cut here ]------------
Badness at crypto/algapi.c:293
Modules linked in: aes_generic aes_s390(+) qeth_l3 qeth_l2 binfmt_misc zfcp qeth ccwgroup
CPU: 5 Tainted: G        W  2.6.28 #25
Process modprobe (pid: 5270, task: 000000003c1e5cf8, ksp: 000000003be83728)
Krnl PSW : 0704100180000000 000000000016a2ac (crypto_wait_for_test+0x80/0x8c)
           R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:0 CC:1 PM:0 EA:3
Krnl GPRS: 0000000000000001 000000003be84000 fffffffffffffe00 00000000003aa6c4
           000000000016a28a 000000003be83c00 00000000000476b1 0000000000000000
           0000000000673048 000000003be80000 000003e000000000 000000003be83bf8
           000000003be28c00 00000000002ffd20 000000000016a28a 000000003be83bf8
Krnl Code: 000000000016a2a0: ebbfb0a00004	lmg	%r11,%r15,160(%r11)
           000000000016a2a6: 07f4		bcr	15,%r4
           000000000016a2a8: a7f40001		brc	15,16a2aa
          >000000000016a2ac: a7f4fff2		brc	15,16a290
           000000000016a2b0: a7f40001		brc	15,16a2b2
           000000000016a2b4: a7f4ffee		brc	15,16a290
           000000000016a2b8: ebbff0700024	stmg	%r11,%r15,112(%r15)
           000000000016a2be: a7f13f00		tmll	%r15,16128
Call Trace:
([<000000000016a28a>] crypto_wait_for_test+0x5e/0x8c)
 [<000000000016a598>] crypto_register_alg+0x84/0x9c
 [<000003e000bce180>] aes_s390_init+0x180/0x1f0 [aes_s390]
 [<00000000000120c4>] do_one_initcall+0x44/0x184
 [<00000000000825da>] sys_init_module+0xce/0x1f4
 [<0000000000027a5e>] sysc_noemu+0x10/0x16
 [<000002000011fda2>] 0x2000011fda2
INFO: lockdep is turned off.
Last Breaking-Event-Address:
 [<000000000016a2a8>] crypto_wait_for_test+0x7c/0x8c
------------[ cut here ]------------
Badness at crypto/algapi.c:293
Modules linked in: aes_generic aes_s390(+) qeth_l3 qeth_l2 binfmt_misc zfcp qeth ccwgroup
CPU: 5 Tainted: G        W  2.6.28 #25
Process modprobe (pid: 5270, task: 000000003c1e5cf8, ksp: 000000003be83728)
Krnl PSW : 0704100180000000 000000000016a2ac (crypto_wait_for_test+0x80/0x8c)
           R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:0 CC:1 PM:0 EA:3
Krnl GPRS: 0000000000000001 000000003be84000 fffffffffffffe00 00000000003aa6c4
           000000000016a28a 000000003be83c00 00000000000476b1 0000000000000000
           0000000000673048 000000003be80000 000003e000000000 000000003be83bf8
           000000003be28c00 00000000002ffd20 000000000016a28a 000000003be83bf8
Krnl Code: 000000000016a2a0: ebbfb0a00004	lmg	%r11,%r15,160(%r11)
           000000000016a2a6: 07f4		bcr	15,%r4
           000000000016a2a8: a7f40001		brc	15,16a2aa
          >000000000016a2ac: a7f4fff2		brc	15,16a290
           000000000016a2b0: a7f40001		brc	15,16a2b2
           000000000016a2b4: a7f4ffee		brc	15,16a290
           000000000016a2b8: ebbff0700024	stmg	%r11,%r15,112(%r15)
           000000000016a2be: a7f13f00		tmll	%r15,16128
Call Trace:
([<000000000016a28a>] crypto_wait_for_test+0x5e/0x8c)
 [<000000000016a598>] crypto_register_alg+0x84/0x9c
 [<000003e000bce192>] aes_s390_init+0x192/0x1f0 [aes_s390]
 [<00000000000120c4>] do_one_initcall+0x44/0x184
 [<00000000000825da>] sys_init_module+0xce/0x1f4
 [<0000000000027a5e>] sysc_noemu+0x10/0x16
 [<000002000011fda2>] 0x2000011fda2

INFO: lockdep is turned off.
Last Breaking-Event-Address:
[<000000000016a2a8>] crypto_wait_for_test+0x7c/0x8c
alg: Unexpected test result for aes-s390: 0

maintainer
#3 lore raw
Quoted 8 lines · Jan Glauber

STACK TRACE FOR TASK: 0x3f6137c8 (modprobe)

STACK:
0 schedule+1136 [0x2df82c]
1 fcntl_setlk+412 [0x107a0c]
2 sys_fcntl+262 [0xd7076]
3 sysc_noemu+16 [0x27a5e]

I see. Please let me know if this patch fixes it.

crypto: api - Fix module load deadlock with fallback algorithms

With the mandatory algorithm testing at registration, we have now created a deadlock with algorithms requiring fallbacks. This can happen if the module containing the algorithm requiring fallback is loaded first, without the fallback module being loaded first. The system will then try to test the new algorithm, find that it needs to load a fallback, and then try to load that.

As both algorithms share the same module alias, it can attempt to load the original algorithm again and block indefinitely.

As algorithms requiring fallbacks are a special case, we can fix this by giving them a different module alias than the rest. Then it's just a matter of using the right aliases according to what algorithms we're trying to find.

Signed-off-by Herbert Xu <[email protected]>
arch/s390/crypto/aes_s390.c +1 -1
index c42cd89..6118890 100644
@@ -556,7 +556,7 @@ static void __exit aes_s390_fini(void)
 module_init(aes_s390_init);
 module_exit(aes_s390_fini);
 
-MODULE_ALIAS("aes");
+MODULE_ALIAS("aes-all");
 
 MODULE_DESCRIPTION("Rijndael (AES) Cipher Algorithm");
 MODULE_LICENSE("GPL");
crypto/api.c +13 -2
index efe77df..38a2bc0 100644
@@ -215,8 +215,19 @@ struct crypto_alg *crypto_larval_lookup(const char *name, u32 type, u32 mask)
 	mask &= ~(CRYPTO_ALG_LARVAL | CRYPTO_ALG_DEAD);
 	type &= mask;
 
-	alg = try_then_request_module(crypto_alg_lookup(name, type, mask),
-				      name);
+	alg = crypto_alg_lookup(name, type, mask);
+	if (!alg) {
+		char tmp[CRYPTO_MAX_ALG_NAME];
+
+		request_module(name);
+
+		if (!((type ^ CRYPTO_ALG_NEED_FALLBACK) & mask) &&
+		    snprintf(tmp, sizeof(tmp), "%s-all", name) < sizeof(tmp))
+			request_module(tmp);
+
+		alg = crypto_alg_lookup(name, type, mask);
+	}
+
 	if (alg)
 		return crypto_is_larval(alg) ? crypto_larval_wait(alg) : alg;
drivers/crypto/padlock-aes.c +1 -1
index 856b3cc..3f0fdd1 100644
@@ -489,4 +489,4 @@ MODULE_DESCRIPTION("VIA PadLock AES algorithm support");
 MODULE_LICENSE("GPL");
 MODULE_AUTHOR("Michal Ludvig");
 
-MODULE_ALIAS("aes");
+MODULE_ALIAS("aes-all");
drivers/crypto/padlock-sha.c +2 -2
index a7fbade..a2c8e85 100644
@@ -304,7 +304,7 @@ MODULE_DESCRIPTION("VIA PadLock SHA1/SHA256 algorithms support.");
 MODULE_LICENSE("GPL");
 MODULE_AUTHOR("Michal Ludvig");
 
-MODULE_ALIAS("sha1");
-MODULE_ALIAS("sha256");
+MODULE_ALIAS("sha1-all");
+MODULE_ALIAS("sha256-all");
 MODULE_ALIAS("sha1-padlock");
 MODULE_ALIAS("sha256-padlock");

Thanks,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
author
#4 lore raw

Hi Herbert,

your patch solves the hanging modprobe (tested on top of cryptodev-2.6). Both modules (aes_generic and aes_s390) are loaded after the modprobe aes_s390.

Thanks a lot, Jan

Quoted 96 lines · Herbert Xu
Quoted 8 lines · Jan Glauber

STACK TRACE FOR TASK: 0x3f6137c8 (modprobe)

STACK:
0 schedule+1136 [0x2df82c]
1 fcntl_setlk+412 [0x107a0c]
2 sys_fcntl+262 [0xd7076]
3 sysc_noemu+16 [0x27a5e]

I see. Please let me know if this patch fixes it.

crypto: api - Fix module load deadlock with fallback algorithms

With the mandatory algorithm testing at registration, we have now created a deadlock with algorithms requiring fallbacks. This can happen if the module containing the algorithm requiring fallback is loaded first, without the fallback module being loaded first. The system will then try to test the new algorithm, find that it needs to load a fallback, and then try to load that.

As both algorithms share the same module alias, it can attempt to load the original algorithm again and block indefinitely.

As algorithms requiring fallbacks are a special case, we can fix this by giving them a different module alias than the rest. Then it's just a matter of using the right aliases according to what algorithms we're trying to find.

Signed-off-by Herbert Xu <[email protected]>
arch/s390/crypto/aes_s390.c +1 -1
index c42cd89..6118890 100644
@@ -556,7 +556,7 @@ static void __exit aes_s390_fini(void)
 module_init(aes_s390_init);
 module_exit(aes_s390_fini);
 
-MODULE_ALIAS("aes");
+MODULE_ALIAS("aes-all");
 
 MODULE_DESCRIPTION("Rijndael (AES) Cipher Algorithm");
 MODULE_LICENSE("GPL");
crypto/api.c +13 -2
index efe77df..38a2bc0 100644
@@ -215,8 +215,19 @@ struct crypto_alg *crypto_larval_lookup(const char *name, u32 type, u32 mask)
 	mask &= ~(CRYPTO_ALG_LARVAL | CRYPTO_ALG_DEAD);
 	type &= mask;
 
-	alg = try_then_request_module(crypto_alg_lookup(name, type, mask),
-				      name);
+	alg = crypto_alg_lookup(name, type, mask);
+	if (!alg) {
+		char tmp[CRYPTO_MAX_ALG_NAME];
+
+		request_module(name);
+
+		if (!((type ^ CRYPTO_ALG_NEED_FALLBACK) & mask) &&
+		    snprintf(tmp, sizeof(tmp), "%s-all", name) < sizeof(tmp))
+			request_module(tmp);
+
+		alg = crypto_alg_lookup(name, type, mask);
+	}
+
 	if (alg)
 		return crypto_is_larval(alg) ? crypto_larval_wait(alg) : alg;
drivers/crypto/padlock-aes.c +1 -1
index 856b3cc..3f0fdd1 100644
@@ -489,4 +489,4 @@ MODULE_DESCRIPTION("VIA PadLock AES algorithm support");
 MODULE_LICENSE("GPL");
 MODULE_AUTHOR("Michal Ludvig");
 
-MODULE_ALIAS("aes");
+MODULE_ALIAS("aes-all");
drivers/crypto/padlock-sha.c +2 -2
index a7fbade..a2c8e85 100644
@@ -304,7 +304,7 @@ MODULE_DESCRIPTION("VIA PadLock SHA1/SHA256 algorithms support.");
 MODULE_LICENSE("GPL");
 MODULE_AUTHOR("Michal Ludvig");
 
-MODULE_ALIAS("sha1");
-MODULE_ALIAS("sha256");
+MODULE_ALIAS("sha1-all");
+MODULE_ALIAS("sha256-all");
 MODULE_ALIAS("sha1-padlock");
 MODULE_ALIAS("sha256-padlock");

Thanks,

#5 lore raw
In reply to Herbert Xu
Quoted 4 lines · * Herbert Xu | 2009-02-26 14:06:18 [+0800]:

As algorithms requiring fallbacks are a special case, we can fix this by giving them a different module alias than the rest. Then it's just a matter of using the right aliases according to what algorithms we're trying to find.

Yup. I am not sure if we have a problem with the geode here. That one is loaded via PCI id probing. I'm going to check on that once I figured out where the devel board is.....

Sebastian

maintainer
#6 lore raw
Quoted 4 lines · Sebastian Andrzej Siewior

Yup. I am not sure if we have a problem with the geode here. That one is loaded via PCI id probing. I'm going to check on that once I figured out where the devel board is.....

Well not everybody uses udev :)

Cheers,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt