Skip to contentSkip to conversation

gss_destroy crash

PATCH In review J. Bruce Fields 2 participants 4 messages Sep 17, 2013 – Sep 18, 2013

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

As of eb6dc19d8e72ce3a957af5511d20c0db0a8bd007 "RPCSEC_GSS: Share all credential caches on a per-transport basis" I'm getting an occasional crash on shutdown of nfsd's 4.0 callback client (see appended oops), because rpc_shutdown_client() is called on a client whose cl_auth is a gss_auth with gss_pipe[0]->clnt pointing to freed memory.

Is there any known bug here?

While I try to understand this....

I'm wondering what guarantees that gss_pipe[0]->clnt is still good at this point, and that leads me to be suspicious of the sharing introduced by this patch--how do we know that a gss_auth can't be shared between two clients that could disappear in either order?

The chasing of cl_parent pointers in gss_create() suggests that the auth's pointers back to clients are always supposed to be back to a common ancestor, but does gss_auth_find_or_add_hashed really guarantee that the auth will only be shared between clients that are cloned from a common ancestor?

--b.

general protection fault: 0000 [#1] PREEMPT SMP 
Modules linked in: rpcsec_gss_krb5 nfsd auth_rpcgss oid_registry nfs_acl lockd sunrpc
CPU: 3 PID: 4071 Comm: kworker/u8:2 Not tainted 3.11.0-rc2-00182-g025145f #1665
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
Workqueue: nfsd4_callbacks nfsd4_do_callback_rpc [nfsd]
task: ffff88003e206080 ti: ffff88003c384000 task.ti: ffff88003c384000
RIP: 0010:[<ffffffffa00001f3>]  [<ffffffffa00001f3>] rpc_net_ns+0x53/0x70 [sunrpc]
RSP: 0000:ffff88003c385ab8  EFLAGS: 00010246
RAX: 6b6b6b6b6b6b6b6b RBX: ffff88003af9a800 RCX: 0000000000000002
RDX: ffffffffa00001a5 RSI: 0000000000000001 RDI: ffffffff81e284e0
RBP: ffff88003c385ad8 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000015 R12: ffff88003c990840
R13: ffff88003c990878 R14: ffff88003c385ba8 R15: ffff88003e206080
FS:  0000000000000000(0000) GS:ffff88003fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fcdf737e000 CR3: 000000003ad2b000 CR4: 00000000000006e0
Stack:
 ffffffffa00001a5 0000000000000006 0000000000000006 ffff88003af9a800
 ffff88003c385b08 ffffffffa00d52a4 ffff88003c385ba8 ffff88003c751bd8
 ffff88003c751bc0 ffff88003e113600 ffff88003c385b18 ffffffffa00d530c
Call Trace:
 [<ffffffffa00001a5>] ? rpc_net_ns+0x5/0x70 [sunrpc]
 [<ffffffffa00d52a4>] __gss_pipe_release+0x54/0x90 [auth_rpcgss]
 [<ffffffffa00d530c>] gss_pipe_free+0x2c/0x30 [auth_rpcgss]
 [<ffffffffa00d678b>] gss_destroy+0x9b/0xf0 [auth_rpcgss]
 [<ffffffffa000de63>] rpcauth_release+0x23/0x30 [sunrpc]
 [<ffffffffa0001e81>] rpc_release_client+0x51/0xb0 [sunrpc]
 [<ffffffffa00020d5>] rpc_shutdown_client+0xe5/0x170 [sunrpc]
 [<ffffffff81098a14>] ? cpuacct_charge+0xa4/0xb0
 [<ffffffff81098975>] ? cpuacct_charge+0x5/0xb0
 [<ffffffffa019556f>] nfsd4_process_cb_update.isra.17+0x2f/0x210 [nfsd]
 [<ffffffff819a4ac0>] ? _raw_spin_unlock_irq+0x30/0x60
 [<ffffffff819a4acb>] ? _raw_spin_unlock_irq+0x3b/0x60
 [<ffffffff810703ab>] ? process_one_work+0x15b/0x510
 [<ffffffffa01957dd>] nfsd4_do_callback_rpc+0x8d/0xa0 [nfsd]
 [<ffffffff8107041e>] process_one_work+0x1ce/0x510
 [<ffffffff810703ab>] ? process_one_work+0x15b/0x510
 [<ffffffff810712ab>] worker_thread+0x11b/0x370
 [<ffffffff81071190>] ? manage_workers.isra.24+0x2b0/0x2b0
 [<ffffffff8107854b>] kthread+0xdb/0xe0
 [<ffffffff819a4ac0>] ? _raw_spin_unlock_irq+0x30/0x60
 [<ffffffff81078470>] ? __init_kthread_worker+0x70/0x70
 [<ffffffff819ac7dc>] ret_from_fork+0x7c/0xb0
 [<ffffffff81078470>] ? __init_kthread_worker+0x70/0x70
Code: a5 01 00 a0 31 d2 31 f6 48 c7 c7 e0 84 e2 81 e8 f4 91 0a e1 48 8b 43 60 48 c7 c2 a5 01 00 a0 be 01 00 00 00 48 c7 c7 e0 84 e2 81 <48> 8b 98 10 07 00 00 e8 91 8f 0a e1 e8 3c 4e 07 e1 48 83 c4 18 
RIP  [<ffffffffa00001f3>] rpc_net_ns+0x53/0x70 [sunrpc]
 RSP <ffff88003c385ab8>
---[ end trace ce5e3f2a85c100c0 ]---
#1 lore raw
Quoted 15 lines · bfields

As of eb6dc19d8e72ce3a957af5511d20c0db0a8bd007 "RPCSEC_GSS: Share all credential caches on a per-transport basis" I'm getting an occasional crash on shutdown of nfsd's 4.0 callback client (see appended oops), because rpc_shutdown_client() is called on a client whose cl_auth is a gss_auth with gss_pipe[0]->clnt pointing to freed memory.

Is there any known bug here?

While I try to understand this....

I'm wondering what guarantees that gss_pipe[0]->clnt is still good at this point, and that leads me to be suspicious of the sharing introduced by this patch--how do we know that a gss_auth can't be shared between two clients that could disappear in either order?

Quoted 5 lines

The chasing of cl_parent pointers in gss_create() suggests that the auth's pointers back to clients are always supposed to be back to a common ancestor, but does gss_auth_find_or_add_hashed really guarantee that the auth will only be shared between clients that are cloned from a common ancestor?

Confirmed that adding a check like

@@ -1085,6 +1085,8 @@ gss_auth_find_or_add_hashed(struct rpc_auth_create_args *args,
gss_auth,
hash,
hashval) {
+ if (gss_auth->client != clnt)
+ continue;
if (gss_auth->rpc_auth.au_flavor != args->pseudoflavor)
continue;
if (gss_auth->target_name != args->target_name) {

fixes the problem. I can send in a proper patch if you think that makes sense....

--b.

Quoted 52 lines

--b.

general protection fault: 0000 [#1] PREEMPT SMP 
Modules linked in: rpcsec_gss_krb5 nfsd auth_rpcgss oid_registry nfs_acl lockd sunrpc
CPU: 3 PID: 4071 Comm: kworker/u8:2 Not tainted 3.11.0-rc2-00182-g025145f #1665
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
Workqueue: nfsd4_callbacks nfsd4_do_callback_rpc [nfsd]
task: ffff88003e206080 ti: ffff88003c384000 task.ti: ffff88003c384000
RIP: 0010:[<ffffffffa00001f3>]  [<ffffffffa00001f3>] rpc_net_ns+0x53/0x70 [sunrpc]
RSP: 0000:ffff88003c385ab8  EFLAGS: 00010246
RAX: 6b6b6b6b6b6b6b6b RBX: ffff88003af9a800 RCX: 0000000000000002
RDX: ffffffffa00001a5 RSI: 0000000000000001 RDI: ffffffff81e284e0
RBP: ffff88003c385ad8 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000015 R12: ffff88003c990840
R13: ffff88003c990878 R14: ffff88003c385ba8 R15: ffff88003e206080
FS:  0000000000000000(0000) GS:ffff88003fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fcdf737e000 CR3: 000000003ad2b000 CR4: 00000000000006e0
Stack:
 ffffffffa00001a5 0000000000000006 0000000000000006 ffff88003af9a800
 ffff88003c385b08 ffffffffa00d52a4 ffff88003c385ba8 ffff88003c751bd8
 ffff88003c751bc0 ffff88003e113600 ffff88003c385b18 ffffffffa00d530c
Call Trace:
 [<ffffffffa00001a5>] ? rpc_net_ns+0x5/0x70 [sunrpc]
 [<ffffffffa00d52a4>] __gss_pipe_release+0x54/0x90 [auth_rpcgss]
 [<ffffffffa00d530c>] gss_pipe_free+0x2c/0x30 [auth_rpcgss]
 [<ffffffffa00d678b>] gss_destroy+0x9b/0xf0 [auth_rpcgss]
 [<ffffffffa000de63>] rpcauth_release+0x23/0x30 [sunrpc]
 [<ffffffffa0001e81>] rpc_release_client+0x51/0xb0 [sunrpc]
 [<ffffffffa00020d5>] rpc_shutdown_client+0xe5/0x170 [sunrpc]
 [<ffffffff81098a14>] ? cpuacct_charge+0xa4/0xb0
 [<ffffffff81098975>] ? cpuacct_charge+0x5/0xb0
 [<ffffffffa019556f>] nfsd4_process_cb_update.isra.17+0x2f/0x210 [nfsd]
 [<ffffffff819a4ac0>] ? _raw_spin_unlock_irq+0x30/0x60
 [<ffffffff819a4acb>] ? _raw_spin_unlock_irq+0x3b/0x60
 [<ffffffff810703ab>] ? process_one_work+0x15b/0x510
 [<ffffffffa01957dd>] nfsd4_do_callback_rpc+0x8d/0xa0 [nfsd]
 [<ffffffff8107041e>] process_one_work+0x1ce/0x510
 [<ffffffff810703ab>] ? process_one_work+0x15b/0x510
 [<ffffffff810712ab>] worker_thread+0x11b/0x370
 [<ffffffff81071190>] ? manage_workers.isra.24+0x2b0/0x2b0
 [<ffffffff8107854b>] kthread+0xdb/0xe0
 [<ffffffff819a4ac0>] ? _raw_spin_unlock_irq+0x30/0x60
 [<ffffffff81078470>] ? __init_kthread_worker+0x70/0x70
 [<ffffffff819ac7dc>] ret_from_fork+0x7c/0xb0
 [<ffffffff81078470>] ? __init_kthread_worker+0x70/0x70
Code: a5 01 00 a0 31 d2 31 f6 48 c7 c7 e0 84 e2 81 e8 f4 91 0a e1 48 8b 43 60 48 c7 c2 a5 01 00 a0 be 01 00 00 00 48 c7 c7 e0 84 e2 81 <48> 8b 98 10 07 00 00 e8 91 8f 0a e1 e8 3c 4e 07 e1 48 83 c4 18 
RIP  [<ffffffffa00001f3>] rpc_net_ns+0x53/0x70 [sunrpc]
 RSP <ffff88003c385ab8>
---[ end trace ce5e3f2a85c100c0 ]---
#2 lore raw

PATCH 1/1 RPCSEC_GSS: fix crash on destroying gss auth

From: "J. Bruce Fields" <[email protected]>

This fixes a regression since eb6dc19d8e72ce3a957af5511d20c0db0a8bd007 "RPCSEC_GSS: Share all credential caches on a per-transport basis" which could cause an occasional oops in the nfsd code (see below).

The problem was that an auth was left referencing a client that had been freed. To avoid this we need to ensure that auths are shared only between descendants of a common client; the fact that a clone of an rpc_client takes a reference on its parent then ensures that the parent client will last as long as the auth.

Also add a comment explaining what I think was the intention of this code.

  general protection fault: 0000 [#1] PREEMPT SMP
  Modules linked in: rpcsec_gss_krb5 nfsd auth_rpcgss oid_registry nfs_acl lockd sunrpc
  CPU: 3 PID: 4071 Comm: kworker/u8:2 Not tainted 3.11.0-rc2-00182-g025145f #1665
  Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
  Workqueue: nfsd4_callbacks nfsd4_do_callback_rpc [nfsd]
  task: ffff88003e206080 ti: ffff88003c384000 task.ti: ffff88003c384000
  RIP: 0010:[<ffffffffa00001f3>]  [<ffffffffa00001f3>] rpc_net_ns+0x53/0x70 [sunrpc]
  RSP: 0000:ffff88003c385ab8  EFLAGS: 00010246
  RAX: 6b6b6b6b6b6b6b6b RBX: ffff88003af9a800 RCX: 0000000000000002
  RDX: ffffffffa00001a5 RSI: 0000000000000001 RDI: ffffffff81e284e0
  RBP: ffff88003c385ad8 R08: 0000000000000001 R09: 0000000000000000
  R10: 0000000000000000 R11: 0000000000000015 R12: ffff88003c990840
  R13: ffff88003c990878 R14: ffff88003c385ba8 R15: ffff88003e206080
  FS:  0000000000000000(0000) GS:ffff88003fd80000(0000) knlGS:0000000000000000
  CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  CR2: 00007fcdf737e000 CR3: 000000003ad2b000 CR4: 00000000000006e0
  Stack:
   ffffffffa00001a5 0000000000000006 0000000000000006 ffff88003af9a800
   ffff88003c385b08 ffffffffa00d52a4 ffff88003c385ba8 ffff88003c751bd8
   ffff88003c751bc0 ffff88003e113600 ffff88003c385b18 ffffffffa00d530c
  Call Trace:
   [<ffffffffa00001a5>] ? rpc_net_ns+0x5/0x70 [sunrpc]
   [<ffffffffa00d52a4>] __gss_pipe_release+0x54/0x90 [auth_rpcgss]
   [<ffffffffa00d530c>] gss_pipe_free+0x2c/0x30 [auth_rpcgss]
   [<ffffffffa00d678b>] gss_destroy+0x9b/0xf0 [auth_rpcgss]
   [<ffffffffa000de63>] rpcauth_release+0x23/0x30 [sunrpc]
   [<ffffffffa0001e81>] rpc_release_client+0x51/0xb0 [sunrpc]
   [<ffffffffa00020d5>] rpc_shutdown_client+0xe5/0x170 [sunrpc]
   [<ffffffff81098a14>] ? cpuacct_charge+0xa4/0xb0
   [<ffffffff81098975>] ? cpuacct_charge+0x5/0xb0
   [<ffffffffa019556f>] nfsd4_process_cb_update.isra.17+0x2f/0x210 [nfsd]
   [<ffffffff819a4ac0>] ? _raw_spin_unlock_irq+0x30/0x60
   [<ffffffff819a4acb>] ? _raw_spin_unlock_irq+0x3b/0x60
   [<ffffffff810703ab>] ? process_one_work+0x15b/0x510
   [<ffffffffa01957dd>] nfsd4_do_callback_rpc+0x8d/0xa0 [nfsd]
   [<ffffffff8107041e>] process_one_work+0x1ce/0x510
   [<ffffffff810703ab>] ? process_one_work+0x15b/0x510
   [<ffffffff810712ab>] worker_thread+0x11b/0x370
   [<ffffffff81071190>] ? manage_workers.isra.24+0x2b0/0x2b0
   [<ffffffff8107854b>] kthread+0xdb/0xe0
   [<ffffffff819a4ac0>] ? _raw_spin_unlock_irq+0x30/0x60
   [<ffffffff81078470>] ? __init_kthread_worker+0x70/0x70
   [<ffffffff819ac7dc>] ret_from_fork+0x7c/0xb0
   [<ffffffff81078470>] ? __init_kthread_worker+0x70/0x70
  Code: a5 01 00 a0 31 d2 31 f6 48 c7 c7 e0 84 e2 81 e8 f4 91 0a e1 48 8b 43 60 48 c7 c2 a5 01 00 a0 be 01 00 00 00 48 c7 c7 e0 84 e2 81 <48> 8b 98 10 07 00 00 e8 91 8f 0a e1 e8
  +3c 4e 07 e1 48 83 c4 18
  RIP  [<ffffffffa00001f3>] rpc_net_ns+0x53/0x70 [sunrpc]
   RSP <ffff88003c385ab8>
---
 net/sunrpc/auth_gss/auth_gss.c |   11 +++++++++++
 1 file changed, 11 insertions(+)
Quoted 2 lines · J. Bruce Fields

fixes the problem. I can send in a proper patch if you think that makes sense....

Here's my attempt.

net/sunrpc/auth_gss/auth_gss.c +11 -0
index fcac5d1..0846566 100644
@@ -1075,6 +1075,15 @@ gss_destroy(struct rpc_auth *auth)
 	kref_put(&gss_auth->kref, gss_free_callback);
 }
 
+/*
+ * Auths may be shared between rpc clients that were cloned from a
+ * common client with the same xprt, if they also share the flavor and
+ * target_name.
+ *
+ * The auth is looked up from the oldest parent sharing the same
+ * cl_xprt, and the auth itself references only that common parent
+ * (which is guaranteed to last as long as any of its descendants).
+ */
 static struct gss_auth *
 gss_auth_find_or_add_hashed(struct rpc_auth_create_args *args,
 		struct rpc_clnt *clnt,
@@ -1088,6 +1097,8 @@ gss_auth_find_or_add_hashed(struct rpc_auth_create_args *args,
 			gss_auth,
 			hash,
 			hashval) {
+		if (gss_auth->client != clnt)
+			continue;
 		if (gss_auth->rpc_auth.au_flavor != args->pseudoflavor)
 			continue;
 		if (gss_auth->target_name != args->target_name) {
Signature · 1 line
1.7.9.5
#3 lore raw

Re: [PATCH] RPCSEC_GSS: fix crash on destroying gss auth

T24gV2VkLCAyMDEzLTA5LTE4IGF0IDExOjE2IC0wNDAwLCBKLiBCcnVjZSBGaWVsZHMgd3JvdGU6 DQo+IEZyb206ICJKLiBCcnVjZSBGaWVsZHMiIDxiZmllbGRzQHJlZGhhdC5jb20+DQo+IA0KPiBU aGlzIGZpeGVzIGEgcmVncmVzc2lvbiBzaW5jZSAgZWI2ZGMxOWQ4ZTcyY2UzYTk1N2FmNTUxMWQy MGMwZGIwYThiZDAwNw0KPiAiUlBDU0VDX0dTUzogU2hhcmUgYWxsIGNyZWRlbnRpYWwgY2FjaGVz IG9uIGEgcGVyLXRyYW5zcG9ydCBiYXNpcyIgd2hpY2gNCj4gY291bGQgY2F1c2UgYW4gb2NjYXNp b25hbCBvb3BzIGluIHRoZSBuZnNkIGNvZGUgKHNlZSBiZWxvdykuDQo+IA0KPiBUaGUgcHJvYmxl bSB3YXMgdGhhdCBhbiBhdXRoIHdhcyBsZWZ0IHJlZmVyZW5jaW5nIGEgY2xpZW50IHRoYXQgaGFk IGJlZW4NCj4gZnJlZWQuICBUbyBhdm9pZCB0aGlzIHdlIG5lZWQgdG8gZW5zdXJlIHRoYXQgYXV0 aHMgYXJlIHNoYXJlZCBvbmx5DQo+IGJldHdlZW4gZGVzY2VuZGFudHMgb2YgYSBjb21tb24gY2xp ZW50OyB0aGUgZmFjdCB0aGF0IGEgY2xvbmUgb2YgYW4NCj4gcnBjX2NsaWVudCB0YWtlcyBhIHJl ZmVyZW5jZSBvbiBpdHMgcGFyZW50IHRoZW4gZW5zdXJlcyB0aGF0IHRoZSBwYXJlbnQNCj4gY2xp ZW50IHdpbGwgbGFzdCBhcyBsb25nIGFzIHRoZSBhdXRoLg0KPiANCj4gQWxzbyBhZGQgYSBjb21t ZW50IGV4cGxhaW5pbmcgd2hhdCBJIHRoaW5rIHdhcyB0aGUgaW50ZW50aW9uIG9mIHRoaXMNCj4g Y29kZS4NCj4gDQo+ICAgZ2VuZXJhbCBwcm90ZWN0aW9uIGZhdWx0OiAwMDAwIFsjMV0gUFJFRU1Q VCBTTVANCj4gICBNb2R1bGVzIGxpbmtlZCBpbjogcnBjc2VjX2dzc19rcmI1IG5mc2QgYXV0aF9y cGNnc3Mgb2lkX3JlZ2lzdHJ5IG5mc19hY2wgbG9ja2Qgc3VucnBjDQo+ICAgQ1BVOiAzIFBJRDog NDA3MSBDb21tOiBrd29ya2VyL3U4OjIgTm90IHRhaW50ZWQgMy4xMS4wLXJjMi0wMDE4Mi1nMDI1 MTQ1ZiAjMTY2NQ0KPiAgIEhhcmR3YXJlIG5hbWU6IEJvY2hzIEJvY2hzLCBCSU9TIEJvY2hzIDAx LzAxLzIwMTENCj4gICBXb3JrcXVldWU6IG5mc2Q0X2NhbGxiYWNrcyBuZnNkNF9kb19jYWxsYmFj a19ycGMgW25mc2RdDQo+ICAgdGFzazogZmZmZjg4MDAzZTIwNjA4MCB0aTogZmZmZjg4MDAzYzM4 NDAwMCB0YXNrLnRpOiBmZmZmODgwMDNjMzg0MDAwDQo+ICAgUklQOiAwMDEwOls8ZmZmZmZmZmZh MDAwMDFmMz5dICBbPGZmZmZmZmZmYTAwMDAxZjM+XSBycGNfbmV0X25zKzB4NTMvMHg3MCBbc3Vu cnBjXQ0KPiAgIFJTUDogMDAwMDpmZmZmODgwMDNjMzg1YWI4ICBFRkxBR1M6IDAwMDEwMjQ2DQo+ ICAgUkFYOiA2YjZiNmI2YjZiNmI2YjZiIFJCWDogZmZmZjg4MDAzYWY5YTgwMCBSQ1g6IDAwMDAw MDAwMDAwMDAwMDINCj4gICBSRFg6IGZmZmZmZmZmYTAwMDAxYTUgUlNJOiAwMDAwMDAwMDAwMDAw MDAxIFJESTogZmZmZmZmZmY4MWUyODRlMA0KPiAgIFJCUDogZmZmZjg4MDAzYzM4NWFkOCBSMDg6 IDAwMDAwMDAwMDAwMDAwMDEgUjA5OiAwMDAwMDAwMDAwMDAwMDAwDQo+ICAgUjEwOiAwMDAwMDAw MDAwMDAwMDAwIFIxMTogMDAwMDAwMDAwMDAwMDAxNSBSMTI6IGZmZmY4ODAwM2M5OTA4NDANCj4g ICBSMTM6IGZmZmY4ODAwM2M5OTA4NzggUjE0OiBmZmZmODgwMDNjMzg1YmE4IFIxNTogZmZmZjg4 MDAzZTIwNjA4MA0KPiAgIEZTOiAgMDAwMDAwMDAwMDAwMDAwMCgwMDAwKSBHUzpmZmZmODgwMDNm ZDgwMDAwKDAwMDApIGtubEdTOjAwMDAwMDAwMDAwMDAwMDANCj4gICBDUzogIDAwMTAgRFM6IDAw MDAgRVM6IDAwMDAgQ1IwOiAwMDAwMDAwMDgwMDUwMDNiDQo+ICAgQ1IyOiAwMDAwN2ZjZGY3Mzdl MDAwIENSMzogMDAwMDAwMDAzYWQyYjAwMCBDUjQ6IDAwMDAwMDAwMDAwMDA2ZTANCj4gICBTdGFj azoNCj4gICAgZmZmZmZmZmZhMDAwMDFhNSAwMDAwMDAwMDAwMDAwMDA2IDAwMDAwMDAwMDAwMDAw MDYgZmZmZjg4MDAzYWY5YTgwMA0KPiAgICBmZmZmODgwMDNjMzg1YjA4IGZmZmZmZmZmYTAwZDUy YTQgZmZmZjg4MDAzYzM4NWJhOCBmZmZmODgwMDNjNzUxYmQ4DQo+ICAgIGZmZmY4ODAwM2M3NTFi YzAgZmZmZjg4MDAzZTExMzYwMCBmZmZmODgwMDNjMzg1YjE4IGZmZmZmZmZmYTAwZDUzMGMNCj4g ICBDYWxsIFRyYWNlOg0KPiAgICBbPGZmZmZmZmZmYTAwMDAxYTU+XSA/IHJwY19uZXRfbnMrMHg1 LzB4NzAgW3N1bnJwY10NCj4gICAgWzxmZmZmZmZmZmEwMGQ1MmE0Pl0gX19nc3NfcGlwZV9yZWxl YXNlKzB4NTQvMHg5MCBbYXV0aF9ycGNnc3NdDQo+ICAgIFs8ZmZmZmZmZmZhMDBkNTMwYz5dIGdz c19waXBlX2ZyZWUrMHgyYy8weDMwIFthdXRoX3JwY2dzc10NCj4gICAgWzxmZmZmZmZmZmEwMGQ2 NzhiPl0gZ3NzX2Rlc3Ryb3krMHg5Yi8weGYwIFthdXRoX3JwY2dzc10NCj4gICAgWzxmZmZmZmZm ZmEwMDBkZTYzPl0gcnBjYXV0aF9yZWxlYXNlKzB4MjMvMHgzMCBbc3VucnBjXQ0KPiAgICBbPGZm ZmZmZmZmYTAwMDFlODE+XSBycGNfcmVsZWFzZV9jbGllbnQrMHg1MS8weGIwIFtzdW5ycGNdDQo+ ICAgIFs8ZmZmZmZmZmZhMDAwMjBkNT5dIHJwY19zaHV0ZG93bl9jbGllbnQrMHhlNS8weDE3MCBb c3VucnBjXQ0KPiAgICBbPGZmZmZmZmZmODEwOThhMTQ+XSA/IGNwdWFjY3RfY2hhcmdlKzB4YTQv MHhiMA0KPiAgICBbPGZmZmZmZmZmODEwOTg5NzU+XSA/IGNwdWFjY3RfY2hhcmdlKzB4NS8weGIw DQo+ICAgIFs8ZmZmZmZmZmZhMDE5NTU2Zj5dIG5mc2Q0X3Byb2Nlc3NfY2JfdXBkYXRlLmlzcmEu MTcrMHgyZi8weDIxMCBbbmZzZF0NCj4gICAgWzxmZmZmZmZmZjgxOWE0YWMwPl0gPyBfcmF3X3Nw aW5fdW5sb2NrX2lycSsweDMwLzB4NjANCj4gICAgWzxmZmZmZmZmZjgxOWE0YWNiPl0gPyBfcmF3 X3NwaW5fdW5sb2NrX2lycSsweDNiLzB4NjANCj4gICAgWzxmZmZmZmZmZjgxMDcwM2FiPl0gPyBw cm9jZXNzX29uZV93b3JrKzB4MTViLzB4NTEwDQo+ICAgIFs8ZmZmZmZmZmZhMDE5NTdkZD5dIG5m c2Q0X2RvX2NhbGxiYWNrX3JwYysweDhkLzB4YTAgW25mc2RdDQo+ICAgIFs8ZmZmZmZmZmY4MTA3 MDQxZT5dIHByb2Nlc3Nfb25lX3dvcmsrMHgxY2UvMHg1MTANCj4gICAgWzxmZmZmZmZmZjgxMDcw M2FiPl0gPyBwcm9jZXNzX29uZV93b3JrKzB4MTViLzB4NTEwDQo+ICAgIFs8ZmZmZmZmZmY4MTA3 MTJhYj5dIHdvcmtlcl90aHJlYWQrMHgxMWIvMHgzNzANCj4gICAgWzxmZmZmZmZmZjgxMDcxMTkw Pl0gPyBtYW5hZ2Vfd29ya2Vycy5pc3JhLjI0KzB4MmIwLzB4MmIwDQo+ICAgIFs8ZmZmZmZmZmY4 MTA3ODU0Yj5dIGt0aHJlYWQrMHhkYi8weGUwDQo+ICAgIFs8ZmZmZmZmZmY4MTlhNGFjMD5dID8g X3Jhd19zcGluX3VubG9ja19pcnErMHgzMC8weDYwDQo+ICAgIFs8ZmZmZmZmZmY4MTA3ODQ3MD5d ID8gX19pbml0X2t0aHJlYWRfd29ya2VyKzB4NzAvMHg3MA0KPiAgICBbPGZmZmZmZmZmODE5YWM3 ZGM+XSByZXRfZnJvbV9mb3JrKzB4N2MvMHhiMA0KPiAgICBbPGZmZmZmZmZmODEwNzg0NzA+XSA/ IF9faW5pdF9rdGhyZWFkX3dvcmtlcisweDcwLzB4NzANCj4gICBDb2RlOiBhNSAwMSAwMCBhMCAz MSBkMiAzMSBmNiA0OCBjNyBjNyBlMCA4NCBlMiA4MSBlOCBmNCA5MSAwYSBlMSA0OCA4YiA0MyA2 MCA0OCBjNyBjMiBhNSAwMSAwMCBhMCBiZSAwMSAwMCAwMCAwMCA0OCBjNyBjNyBlMCA4NCBlMiA4 MSA8NDg+IDhiIDk4IDEwIDA3IDAwIDAwIGU4IDkxIDhmIDBhIGUxIGU4DQo+ICAgKzNjIDRlIDA3 IGUxIDQ4IDgzIGM0IDE4DQo+ICAgUklQICBbPGZmZmZmZmZmYTAwMDAxZjM+XSBycGNfbmV0X25z KzB4NTMvMHg3MCBbc3VucnBjXQ0KPiAgICBSU1AgPGZmZmY4ODAwM2MzODVhYjg+DQo+IA0KPiBT aWduZWQtb2ZmLWJ5OiBKLiBCcnVjZSBGaWVsZHMgPGJmaWVsZHNAcmVkaGF0LmNvbT4NCj4gLS0t DQo+ICBuZXQvc3VucnBjL2F1dGhfZ3NzL2F1dGhfZ3NzLmMgfCAgIDExICsrKysrKysrKysrDQo+ ICAxIGZpbGUgY2hhbmdlZCwgMTEgaW5zZXJ0aW9ucygrKQ0KPiANCj4gT24gVHVlLCBTZXAgMTcs IDIwMTMgYXQgMDY6MTk6MTRQTSAtMDQwMCwgSi4gQnJ1Y2UgRmllbGRzIHdyb3RlOg0KPiA+IGZp eGVzIHRoZSBwcm9ibGVtLiAgSSBjYW4gc2VuZCBpbiBhIHByb3BlciBwYXRjaCBpZiB5b3UgdGhp bmsgdGhhdCBtYWtlcw0KPiA+IHNlbnNlLi4uLg0KPiANCj4gSGVyZSdzIG15IGF0dGVtcHQuDQoN ClRoYXQgbG9va3MgZ29vZC4gVGhhbmtzIQ0KDQogIFRyb25kDQotLSANClRyb25kIE15a2xlYnVz dA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVidXN0 QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAuY29tDQo=