Skip to contentSkip to conversation

Module removal-related regression?

DISCUSSION Jakub Kicinski 3 participants 20 messages Sep 9, 2017 – Sep 13, 2017

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi!

I'm having trouble with modules on linux/master. rmmod succeeds but the module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
	/opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
	lsmod | grep nfp; \
	rmmod nfp; \
	lsmod | grep nfp 
nfp                   249856  0 
nfp                   200704  1 

If I rmmod again the module will be actually unloaded. The user space is mostly Ubuntu 14.04. Has anyone seen this? I'm trying to bisect now...

#1 lore raw
Quoted 13 lines · Jakub Kicinski

Hi!

I'm having trouble with modules on linux/master. rmmod succeeds but the module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
	/opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
	lsmod | grep nfp; \
	rmmod nfp; \
	lsmod | grep nfp 
nfp                   249856  0 
nfp                   200704  1 
Quoted 3 lines

If I rmmod again the module will be actually unloaded. The user space is mostly Ubuntu 14.04. Has anyone seen this? I'm trying to bisect now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD, refs/bisect/bad)

Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a driver

There are certain touch controllers that may come up in either normal (application) or boot mode, depending on whether firmware/configuration is corrupted when they are powered on. In boot mode the kernel does not create input device instance (because it does not necessarily know the characteristics of the input device in question).

Another number of controllers does not store firmware in a non-volatile memory, and they similarly need to have firmware loaded before input device instance is created. There are also other types of devices with similar behavior.

There is a desire to be able to trigger firmware loading via udev, but it has to happen only when driver is bound to a physical device (i2c or spi). These udev actions can not use ADD events, as those happen too early, so we are introducing BIND and UNBIND events that are emitted at the right moment.

Also, many drivers create additional driver-specific device attributes when binding to the device, to provide userspace with additional controls. The new events allow userspace to adjust these driver-specific attributes without worrying that they are not there yet.

Heeello... :)

#2 lore raw
Quoted 18 lines · Jakub Kicinski
Quoted 13 lines · Jakub Kicinski

Hi!

I'm having trouble with modules on linux/master. rmmod succeeds but the module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1
Quoted 3 lines

If I rmmod again the module will be actually unloaded. The user space is mostly Ubuntu 14.04. Has anyone seen this? I'm trying to bisect now...

Quoted 7 lines

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD, refs/bisect/bad)

Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a driver

Does it happen with all modules or only nfp one?

It seems to work here:

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

Thanks.

Signature · 1 line
Dmitry
#3 lore raw
Quoted 29 lines · Dmitry Torokhov
Quoted 18 lines · Jakub Kicinski
Quoted 13 lines · Jakub Kicinski

Hi!

I'm having trouble with modules on linux/master. rmmod succeeds but the module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1
Quoted 3 lines

If I rmmod again the module will be actually unloaded. The user space is mostly Ubuntu 14.04. Has anyone seen this? I'm trying to bisect now...

Quoted 7 lines

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD, refs/bisect/bad)

Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a driver

Does it happen with all modules or only nfp one?

Quoted 7 lines

It seems to work here:

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

Yes, and only if FW is loaded/requested successfully. Hmm...

I'm building a kernel with more debug, any particular suggestions?

#4 lore raw
In reply to Dmitry Torokhov
Quoted 29 lines · Dmitry Torokhov
Quoted 18 lines · Jakub Kicinski
Quoted 13 lines · Jakub Kicinski

Hi!

I'm having trouble with modules on linux/master. rmmod succeeds but the module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1
Quoted 3 lines

If I rmmod again the module will be actually unloaded. The user space is mostly Ubuntu 14.04. Has anyone seen this? I'm trying to bisect now...

Quoted 7 lines

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD, refs/bisect/bad)

Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a driver

Does it happen with all modules or only nfp one?

Quoted 7 lines

It seems to work here:

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded. The driver used to return EPROBE_DEFER, but I think it doesn't any more (rebuilding the kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

#5 lore raw
Quoted 43 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines · Jakub Kicinski
Quoted 3 lines

Hi!

I'm having trouble with modules on linux/master. rmmod succeeds

but the

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1

If I rmmod again the module will be actually unloaded. The user

space

Quoted 1 line
Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying to

bisect

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 7 lines
Quoted 4 lines
Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a driver

Does it happen with all modules or only nfp one?

Quoted 7 lines

It seems to work here:

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

Quoted 6 lines

It looks like the driver is actually reloaded. The driver used to return EPROBE_DEFER, but I think it doesn't any more (rebuilding the kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

It depends on how silly the udev rules are, but yes, this can definitely happen.

Thanks.

Signature · 1 line
Dmitry
#6 lore raw
Quoted 44 lines · Dmitry Torokhov
Quoted 43 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines · Jakub Kicinski
Quoted 3 lines

Hi!

I'm having trouble with modules on linux/master. rmmod succeeds

but the

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1

If I rmmod again the module will be actually unloaded. The user

space

Quoted 1 line
Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying to

bisect

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 14 lines
Quoted 4 lines
Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a driver

Does it happen with all modules or only nfp one?

It seems to work here:

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

Quoted 8 lines
Quoted 6 lines

It looks like the driver is actually reloaded. The driver used to return EPROBE_DEFER, but I think it doesn't any more (rebuilding the kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

It depends on how silly the udev rules are, but yes, this can definitely happen.

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$

I tested without any udev rules in /etc/udev/, just the standard distro ones. Same thing.

Please find attached the logs from rmmod with kobject debug enabled.

[  155.670272] bus: 'pci': remove driver nfp_netvf
[  155.670398] kobject: 'nfp_netvf' (ffff8807384510a8): kobject_cleanup, parent ffff88045331bbe8
[  155.670405] kobject: 'nfp_netvf' (ffff8807384510a8): auto cleanup 'remove' event
[  155.670412] kobject: 'nfp_netvf' (ffff8807384510a8): kobject_uevent_env
[  155.670480] kobject: 'nfp_netvf' (ffff8807384510a8): fill_kobj_path: path = '/bus/pci/drivers/nfp_netvf'
[  155.670587] kobject: 'nfp_netvf' (ffff8807384510a8): auto cleanup kobject_del
[  155.670620] kobject: 'nfp_netvf' (ffff8807384510a8): calling ktype release
[  155.670627] driver: 'nfp_netvf': driver_release
[  155.670642] kobject: 'nfp_netvf': free name
[  155.670727] bus: 'pci': remove driver nfp
[  155.670795] device: 'hwmon0': device_unregister
[  155.671127] PM: Removing info for No Bus:hwmon0
[  155.671144] kobject: 'hwmon0' (ffff88075ce44b20): kobject_uevent_env
[  155.671219] kobject: 'hwmon0' (ffff88075ce44b20): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/hwmon/hwmon0'
[  155.671329] kobject: 'hwmon' (ffff88074e448fc8): kobject_cleanup, parent ffff880751b480b8
[  155.671336] kobject: 'hwmon' (ffff88074e448fc8): auto cleanup kobject_del
[  155.671369] kobject: 'hwmon' (ffff88074e448fc8): calling ktype release
[  155.671395] kobject: 'hwmon': free name
[  155.671403] kobject: 'hwmon0' (ffff88075ce44b20): kobject_cleanup, parent           (null)
[  155.671409] kobject: 'hwmon0' (ffff88075ce44b20): calling ktype release
[  155.671438] kobject: 'hwmon0': free name
[  156.489986] kobject: 'rx-31' (ffff88074867f7d8): kobject_cleanup, parent ffff88075401a048
[  156.490002] kobject: 'rx-31' (ffff88074867f7d8): auto cleanup 'remove' event
[  156.490009] kobject: 'rx-31' (ffff88074867f7d8): kobject_uevent_env
[  156.490102] kobject: 'rx-31' (ffff88074867f7d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-31'
[  156.490557] kobject: 'rx-31' (ffff88074867f7d8): auto cleanup kobject_del
[  156.490790] kobject: 'rx-31' (ffff88074867f7d8): calling ktype release
[  156.490813] kobject: 'rx-31': free name
[  156.490870] kobject: 'rx-30' (ffff88074867f758): kobject_cleanup, parent ffff88075401a048
[  156.490877] kobject: 'rx-30' (ffff88074867f758): auto cleanup 'remove' event
[  156.490884] kobject: 'rx-30' (ffff88074867f758): kobject_uevent_env
[  156.490992] kobject: 'rx-30' (ffff88074867f758): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-30'
[  156.491124] kobject: 'rx-30' (ffff88074867f758): auto cleanup kobject_del
[  156.491208] kobject: 'rx-30' (ffff88074867f758): calling ktype release
[  156.491214] kobject: 'rx-30': free name
[  156.491235] kobject: 'rx-29' (ffff88074867f6d8): kobject_cleanup, parent ffff88075401a048
[  156.491242] kobject: 'rx-29' (ffff88074867f6d8): auto cleanup 'remove' event
[  156.491249] kobject: 'rx-29' (ffff88074867f6d8): kobject_uevent_env
[  156.491351] kobject: 'rx-29' (ffff88074867f6d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-29'
[  156.491493] kobject: 'rx-29' (ffff88074867f6d8): auto cleanup kobject_del
[  156.491570] kobject: 'rx-29' (ffff88074867f6d8): calling ktype release
[  156.491577] kobject: 'rx-29': free name
[  156.491599] kobject: 'rx-28' (ffff88074867f658): kobject_cleanup, parent ffff88075401a048
[  156.491605] kobject: 'rx-28' (ffff88074867f658): auto cleanup 'remove' event
[  156.491612] kobject: 'rx-28' (ffff88074867f658): kobject_uevent_env
[  156.491707] kobject: 'rx-28' (ffff88074867f658): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-28'
[  156.491844] kobject: 'rx-28' (ffff88074867f658): auto cleanup kobject_del
[  156.491961] kobject: 'rx-28' (ffff88074867f658): calling ktype release
[  156.491967] kobject: 'rx-28': free name
[  156.491996] kobject: 'rx-27' (ffff88074867f5d8): kobject_cleanup, parent ffff88075401a048
[  156.492002] kobject: 'rx-27' (ffff88074867f5d8): auto cleanup 'remove' event
[  156.492009] kobject: 'rx-27' (ffff88074867f5d8): kobject_uevent_env
[  156.492103] kobject: 'rx-27' (ffff88074867f5d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-27'
[  156.492244] kobject: 'rx-27' (ffff88074867f5d8): auto cleanup kobject_del
[  156.492327] kobject: 'rx-27' (ffff88074867f5d8): calling ktype release
[  156.492333] kobject: 'rx-27': free name
[  156.492364] kobject: 'rx-26' (ffff88074867f558): kobject_cleanup, parent ffff88075401a048
[  156.492370] kobject: 'rx-26' (ffff88074867f558): auto cleanup 'remove' event
[  156.492376] kobject: 'rx-26' (ffff88074867f558): kobject_uevent_env
[  156.492481] kobject: 'rx-26' (ffff88074867f558): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-26'
[  156.492618] kobject: 'rx-26' (ffff88074867f558): auto cleanup kobject_del
[  156.492727] kobject: 'rx-26' (ffff88074867f558): calling ktype release
[  156.492733] kobject: 'rx-26': free name
[  156.492764] kobject: 'rx-25' (ffff88074867f4d8): kobject_cleanup, parent ffff88075401a048
[  156.492771] kobject: 'rx-25' (ffff88074867f4d8): auto cleanup 'remove' event
[  156.492777] kobject: 'rx-25' (ffff88074867f4d8): kobject_uevent_env
[  156.492861] kobject: 'rx-25' (ffff88074867f4d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-25'
[  156.493001] kobject: 'rx-25' (ffff88074867f4d8): auto cleanup kobject_del
[  156.493111] kobject: 'rx-25' (ffff88074867f4d8): calling ktype release
[  156.493117] kobject: 'rx-25': free name
[  156.493146] kobject: 'rx-24' (ffff88074867f458): kobject_cleanup, parent ffff88075401a048
[  156.493152] kobject: 'rx-24' (ffff88074867f458): auto cleanup 'remove' event
[  156.493159] kobject: 'rx-24' (ffff88074867f458): kobject_uevent_env
[  156.493287] kobject: 'rx-24' (ffff88074867f458): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-24'
[  156.493428] kobject: 'rx-24' (ffff88074867f458): auto cleanup kobject_del
[  156.493532] kobject: 'rx-24' (ffff88074867f458): calling ktype release
[  156.493538] kobject: 'rx-24': free name
[  156.493567] kobject: 'rx-23' (ffff88074867f3d8): kobject_cleanup, parent ffff88075401a048
[  156.493574] kobject: 'rx-23' (ffff88074867f3d8): auto cleanup 'remove' event
[  156.493580] kobject: 'rx-23' (ffff88074867f3d8): kobject_uevent_env
[  156.493667] kobject: 'rx-23' (ffff88074867f3d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-23'
[  156.493828] kobject: 'rx-23' (ffff88074867f3d8): auto cleanup kobject_del
[  156.493956] kobject: 'rx-23' (ffff88074867f3d8): calling ktype release
[  156.493963] kobject: 'rx-23': free name
[  156.493989] kobject: 'rx-22' (ffff88074867f358): kobject_cleanup, parent ffff88075401a048
[  156.493996] kobject: 'rx-22' (ffff88074867f358): auto cleanup 'remove' event
[  156.494003] kobject: 'rx-22' (ffff88074867f358): kobject_uevent_env
[  156.494084] kobject: 'rx-22' (ffff88074867f358): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-22'
[  156.494240] kobject: 'rx-22' (ffff88074867f358): auto cleanup kobject_del
[  156.494358] kobject: 'rx-22' (ffff88074867f358): calling ktype release
[  156.494365] kobject: 'rx-22': free name
[  156.494390] kobject: 'rx-21' (ffff88074867f2d8): kobject_cleanup, parent ffff88075401a048
[  156.494397] kobject: 'rx-21' (ffff88074867f2d8): auto cleanup 'remove' event
[  156.494404] kobject: 'rx-21' (ffff88074867f2d8): kobject_uevent_env
[  156.494574] kobject: 'rx-21' (ffff88074867f2d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-21'
[  156.494773] kobject: 'rx-21' (ffff88074867f2d8): auto cleanup kobject_del
[  156.494899] kobject: 'rx-21' (ffff88074867f2d8): calling ktype release
[  156.494906] kobject: 'rx-21': free name
[  156.494934] kobject: 'rx-20' (ffff88074867f258): kobject_cleanup, parent ffff88075401a048
[  156.494941] kobject: 'rx-20' (ffff88074867f258): auto cleanup 'remove' event
[  156.494949] kobject: 'rx-20' (ffff88074867f258): kobject_uevent_env
[  156.495062] kobject: 'rx-20' (ffff88074867f258): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-20'
[  156.495243] kobject: 'rx-20' (ffff88074867f258): auto cleanup kobject_del
[  156.495355] kobject: 'rx-20' (ffff88074867f258): calling ktype release
[  156.495362] kobject: 'rx-20': free name
[  156.495388] kobject: 'rx-19' (ffff88074867f1d8): kobject_cleanup, parent ffff88075401a048
[  156.495396] kobject: 'rx-19' (ffff88074867f1d8): auto cleanup 'remove' event
[  156.495404] kobject: 'rx-19' (ffff88074867f1d8): kobject_uevent_env
[  156.495548] kobject: 'rx-19' (ffff88074867f1d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-19'
[  156.495720] kobject: 'rx-19' (ffff88074867f1d8): auto cleanup kobject_del
[  156.495841] kobject: 'rx-19' (ffff88074867f1d8): calling ktype release
[  156.495848] kobject: 'rx-19': free name
[  156.495877] kobject: 'rx-18' (ffff88074867f158): kobject_cleanup, parent ffff88075401a048
[  156.495884] kobject: 'rx-18' (ffff88074867f158): auto cleanup 'remove' event
[  156.495892] kobject: 'rx-18' (ffff88074867f158): kobject_uevent_env
[  156.495994] kobject: 'rx-18' (ffff88074867f158): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-18'
[  156.496178] kobject: 'rx-18' (ffff88074867f158): auto cleanup kobject_del
[  156.496287] kobject: 'rx-18' (ffff88074867f158): calling ktype release
[  156.496308] kobject: 'rx-18': free name
[  156.496337] kobject: 'rx-17' (ffff88074867f0d8): kobject_cleanup, parent ffff88075401a048
[  156.496344] kobject: 'rx-17' (ffff88074867f0d8): auto cleanup 'remove' event
[  156.496351] kobject: 'rx-17' (ffff88074867f0d8): kobject_uevent_env
[  156.496456] kobject: 'rx-17' (ffff88074867f0d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-17'
[  156.496586] kobject: 'rx-17' (ffff88074867f0d8): auto cleanup kobject_del
[  156.496675] kobject: 'rx-17' (ffff88074867f0d8): calling ktype release
[  156.496682] kobject: 'rx-17': free name
[  156.496706] kobject: 'rx-16' (ffff88074867f058): kobject_cleanup, parent ffff88075401a048
[  156.496713] kobject: 'rx-16' (ffff88074867f058): auto cleanup 'remove' event
[  156.496720] kobject: 'rx-16' (ffff88074867f058): kobject_uevent_env
[  156.496801] kobject: 'rx-16' (ffff88074867f058): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-16'
[  156.496942] kobject: 'rx-16' (ffff88074867f058): auto cleanup kobject_del
[  156.497030] kobject: 'rx-16' (ffff88074867f058): calling ktype release
[  156.497037] kobject: 'rx-16': free name
[  156.497061] kobject: 'rx-15' (ffff88074867efd8): kobject_cleanup, parent ffff88075401a048
[  156.497067] kobject: 'rx-15' (ffff88074867efd8): auto cleanup 'remove' event
[  156.497074] kobject: 'rx-15' (ffff88074867efd8): kobject_uevent_env
[  156.497148] kobject: 'rx-15' (ffff88074867efd8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-15'
[  156.497277] kobject: 'rx-15' (ffff88074867efd8): auto cleanup kobject_del
[  156.497358] kobject: 'rx-15' (ffff88074867efd8): calling ktype release
[  156.497365] kobject: 'rx-15': free name
[  156.497393] kobject: 'rx-14' (ffff88074867ef58): kobject_cleanup, parent ffff88075401a048
[  156.497400] kobject: 'rx-14' (ffff88074867ef58): auto cleanup 'remove' event
[  156.497407] kobject: 'rx-14' (ffff88074867ef58): kobject_uevent_env
[  156.497492] kobject: 'rx-14' (ffff88074867ef58): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-14'
[  156.497654] kobject: 'rx-14' (ffff88074867ef58): auto cleanup kobject_del
[  156.497753] kobject: 'rx-14' (ffff88074867ef58): calling ktype release
[  156.497759] kobject: 'rx-14': free name
[  156.497789] kobject: 'rx-13' (ffff88074867eed8): kobject_cleanup, parent ffff88075401a048
[  156.497795] kobject: 'rx-13' (ffff88074867eed8): auto cleanup 'remove' event
[  156.497802] kobject: 'rx-13' (ffff88074867eed8): kobject_uevent_env
[  156.497888] kobject: 'rx-13' (ffff88074867eed8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-13'
[  156.498017] kobject: 'rx-13' (ffff88074867eed8): auto cleanup kobject_del
[  156.498100] kobject: 'rx-13' (ffff88074867eed8): calling ktype release
[  156.498106] kobject: 'rx-13': free name
[  156.498129] kobject: 'rx-12' (ffff88074867ee58): kobject_cleanup, parent ffff88075401a048
[  156.498136] kobject: 'rx-12' (ffff88074867ee58): auto cleanup 'remove' event
[  156.498142] kobject: 'rx-12' (ffff88074867ee58): kobject_uevent_env
[  156.498231] kobject: 'rx-12' (ffff88074867ee58): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-12'
[  156.498357] kobject: 'rx-12' (ffff88074867ee58): auto cleanup kobject_del
[  156.498521] kobject: 'rx-12' (ffff88074867ee58): calling ktype release
[  156.498530] kobject: 'rx-12': free name
[  156.498558] kobject: 'rx-11' (ffff88074867edd8): kobject_cleanup, parent ffff88075401a048
[  156.498570] kobject: 'rx-11' (ffff88074867edd8): auto cleanup 'remove' event
[  156.498581] kobject: 'rx-11' (ffff88074867edd8): kobject_uevent_env
[  156.498683] kobject: 'rx-11' (ffff88074867edd8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-11'
[  156.498905] kobject: 'rx-11' (ffff88074867edd8): auto cleanup kobject_del
[  156.499021] kobject: 'rx-11' (ffff88074867edd8): calling ktype release
[  156.499028] kobject: 'rx-11': free name
[  156.499053] kobject: 'rx-10' (ffff88074867ed58): kobject_cleanup, parent ffff88075401a048
[  156.499060] kobject: 'rx-10' (ffff88074867ed58): auto cleanup 'remove' event
[  156.499066] kobject: 'rx-10' (ffff88074867ed58): kobject_uevent_env
[  156.499149] kobject: 'rx-10' (ffff88074867ed58): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-10'
[  156.499269] kobject: 'rx-10' (ffff88074867ed58): auto cleanup kobject_del
[  156.499374] kobject: 'rx-10' (ffff88074867ed58): calling ktype release
[  156.499380] kobject: 'rx-10': free name
[  156.499411] kobject: 'rx-9' (ffff88074867ecd8): kobject_cleanup, parent ffff88075401a048
[  156.499417] kobject: 'rx-9' (ffff88074867ecd8): auto cleanup 'remove' event
[  156.499424] kobject: 'rx-9' (ffff88074867ecd8): kobject_uevent_env
[  156.499517] kobject: 'rx-9' (ffff88074867ecd8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-9'
[  156.499618] kobject: 'rx-9' (ffff88074867ecd8): auto cleanup kobject_del
[  156.499715] kobject: 'rx-9' (ffff88074867ecd8): calling ktype release
[  156.499721] kobject: 'rx-9': free name
[  156.499748] kobject: 'rx-8' (ffff88074867ec58): kobject_cleanup, parent ffff88075401a048
[  156.499754] kobject: 'rx-8' (ffff88074867ec58): auto cleanup 'remove' event
[  156.499760] kobject: 'rx-8' (ffff88074867ec58): kobject_uevent_env
[  156.499831] kobject: 'rx-8' (ffff88074867ec58): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-8'
[  156.499955] kobject: 'rx-8' (ffff88074867ec58): auto cleanup kobject_del
[  156.500051] kobject: 'rx-8' (ffff88074867ec58): calling ktype release
[  156.500057] kobject: 'rx-8': free name
[  156.500079] kobject: 'rx-7' (ffff88074867ebd8): kobject_cleanup, parent ffff88075401a048
[  156.500086] kobject: 'rx-7' (ffff88074867ebd8): auto cleanup 'remove' event
[  156.500092] kobject: 'rx-7' (ffff88074867ebd8): kobject_uevent_env
[  156.500167] kobject: 'rx-7' (ffff88074867ebd8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-7'
[  156.500290] kobject: 'rx-7' (ffff88074867ebd8): auto cleanup kobject_del
[  156.500364] kobject: 'rx-7' (ffff88074867ebd8): calling ktype release
[  156.500370] kobject: 'rx-7': free name
[  156.500390] kobject: 'rx-6' (ffff88074867eb58): kobject_cleanup, parent ffff88075401a048
[  156.500396] kobject: 'rx-6' (ffff88074867eb58): auto cleanup 'remove' event
[  156.500403] kobject: 'rx-6' (ffff88074867eb58): kobject_uevent_env
[  156.500485] kobject: 'rx-6' (ffff88074867eb58): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-6'
[  156.500608] kobject: 'rx-6' (ffff88074867eb58): auto cleanup kobject_del
[  156.500712] kobject: 'rx-6' (ffff88074867eb58): calling ktype release
[  156.500718] kobject: 'rx-6': free name
[  156.500747] kobject: 'rx-5' (ffff88074867ead8): kobject_cleanup, parent ffff88075401a048
[  156.500753] kobject: 'rx-5' (ffff88074867ead8): auto cleanup 'remove' event
[  156.500760] kobject: 'rx-5' (ffff88074867ead8): kobject_uevent_env
[  156.500842] kobject: 'rx-5' (ffff88074867ead8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-5'
[  156.500956] kobject: 'rx-5' (ffff88074867ead8): auto cleanup kobject_del
[  156.501048] kobject: 'rx-5' (ffff88074867ead8): calling ktype release
[  156.501054] kobject: 'rx-5': free name
[  156.501076] kobject: 'rx-4' (ffff88074867ea58): kobject_cleanup, parent ffff88075401a048
[  156.501083] kobject: 'rx-4' (ffff88074867ea58): auto cleanup 'remove' event
[  156.501089] kobject: 'rx-4' (ffff88074867ea58): kobject_uevent_env
[  156.501179] kobject: 'rx-4' (ffff88074867ea58): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-4'
[  156.501291] kobject: 'rx-4' (ffff88074867ea58): auto cleanup kobject_del
[  156.501378] kobject: 'rx-4' (ffff88074867ea58): calling ktype release
[  156.501384] kobject: 'rx-4': free name
[  156.501415] kobject: 'rx-3' (ffff88074867e9d8): kobject_cleanup, parent ffff88075401a048
[  156.501422] kobject: 'rx-3' (ffff88074867e9d8): auto cleanup 'remove' event
[  156.501428] kobject: 'rx-3' (ffff88074867e9d8): kobject_uevent_env
[  156.501511] kobject: 'rx-3' (ffff88074867e9d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-3'
[  156.501637] kobject: 'rx-3' (ffff88074867e9d8): auto cleanup kobject_del
[  156.501729] kobject: 'rx-3' (ffff88074867e9d8): calling ktype release
[  156.501735] kobject: 'rx-3': free name
[  156.501763] kobject: 'rx-2' (ffff88074867e958): kobject_cleanup, parent ffff88075401a048
[  156.501769] kobject: 'rx-2' (ffff88074867e958): auto cleanup 'remove' event
[  156.501776] kobject: 'rx-2' (ffff88074867e958): kobject_uevent_env
[  156.501870] kobject: 'rx-2' (ffff88074867e958): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-2'
[  156.501980] kobject: 'rx-2' (ffff88074867e958): auto cleanup kobject_del
[  156.502069] kobject: 'rx-2' (ffff88074867e958): calling ktype release
[  156.502076] kobject: 'rx-2': free name
[  156.502104] kobject: 'rx-1' (ffff88074867e8d8): kobject_cleanup, parent ffff88075401a048
[  156.502110] kobject: 'rx-1' (ffff88074867e8d8): auto cleanup 'remove' event
[  156.502117] kobject: 'rx-1' (ffff88074867e8d8): kobject_uevent_env
[  156.502198] kobject: 'rx-1' (ffff88074867e8d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-1'
[  156.502322] kobject: 'rx-1' (ffff88074867e8d8): auto cleanup kobject_del
[  156.502404] kobject: 'rx-1' (ffff88074867e8d8): calling ktype release
[  156.502410] kobject: 'rx-1': free name
[  156.502501] kobject: 'rx-0' (ffff88074867e858): kobject_cleanup, parent ffff88075401a048
[  156.502510] kobject: 'rx-0' (ffff88074867e858): auto cleanup 'remove' event
[  156.502518] kobject: 'rx-0' (ffff88074867e858): kobject_uevent_env
[  156.502641] kobject: 'rx-0' (ffff88074867e858): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/rx-0'
[  156.502851] kobject: 'rx-0' (ffff88074867e858): auto cleanup kobject_del
[  156.502927] kobject: 'rx-0' (ffff88074867e858): calling ktype release
[  156.502934] kobject: 'rx-0': free name
[  156.503119] kobject: 'tx-31' (ffff880735ceee98): kobject_cleanup, parent ffff88075401a048
[  156.503125] kobject: 'tx-31' (ffff880735ceee98): auto cleanup 'remove' event
[  156.503132] kobject: 'tx-31' (ffff880735ceee98): kobject_uevent_env
[  156.503236] kobject: 'tx-31' (ffff880735ceee98): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-31'
[  156.503399] kobject: 'tx-31' (ffff880735ceee98): auto cleanup kobject_del
[  156.503549] kobject: 'tx-31' (ffff880735ceee98): calling ktype release
[  156.503556] kobject: 'tx-31': free name
[  156.503705] kobject: 'tx-30' (ffff880735ceed18): kobject_cleanup, parent ffff88075401a048
[  156.503711] kobject: 'tx-30' (ffff880735ceed18): auto cleanup 'remove' event
[  156.503718] kobject: 'tx-30' (ffff880735ceed18): kobject_uevent_env
[  156.503802] kobject: 'tx-30' (ffff880735ceed18): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-30'
[  156.503924] kobject: 'tx-30' (ffff880735ceed18): auto cleanup kobject_del
[  156.504051] kobject: 'tx-30' (ffff880735ceed18): calling ktype release
[  156.504057] kobject: 'tx-30': free name
[  156.504221] kobject: 'tx-29' (ffff880735ceeb98): kobject_cleanup, parent ffff88075401a048
[  156.504228] kobject: 'tx-29' (ffff880735ceeb98): auto cleanup 'remove' event
[  156.504234] kobject: 'tx-29' (ffff880735ceeb98): kobject_uevent_env
[  156.504323] kobject: 'tx-29' (ffff880735ceeb98): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-29'
[  156.504433] kobject: 'tx-29' (ffff880735ceeb98): auto cleanup kobject_del
[  156.504541] kobject: 'tx-29' (ffff880735ceeb98): calling ktype release
[  156.504547] kobject: 'tx-29': free name
[  156.504694] kobject: 'tx-28' (ffff880735ceea18): kobject_cleanup, parent ffff88075401a048
[  156.504701] kobject: 'tx-28' (ffff880735ceea18): auto cleanup 'remove' event
[  156.504707] kobject: 'tx-28' (ffff880735ceea18): kobject_uevent_env
[  156.504784] kobject: 'tx-28' (ffff880735ceea18): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-28'
[  156.504904] kobject: 'tx-28' (ffff880735ceea18): auto cleanup kobject_del
[  156.505037] kobject: 'tx-28' (ffff880735ceea18): calling ktype release
[  156.505043] kobject: 'tx-28': free name
[  156.505163] kobject: 'tx-27' (ffff880735cee898): kobject_cleanup, parent ffff88075401a048
[  156.505169] kobject: 'tx-27' (ffff880735cee898): auto cleanup 'remove' event
[  156.505176] kobject: 'tx-27' (ffff880735cee898): kobject_uevent_env
[  156.505257] kobject: 'tx-27' (ffff880735cee898): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-27'
[  156.505367] kobject: 'tx-27' (ffff880735cee898): auto cleanup kobject_del
[  156.505475] kobject: 'tx-27' (ffff880735cee898): calling ktype release
[  156.505482] kobject: 'tx-27': free name
[  156.505627] kobject: 'tx-26' (ffff880735cee718): kobject_cleanup, parent ffff88075401a048
[  156.505633] kobject: 'tx-26' (ffff880735cee718): auto cleanup 'remove' event
[  156.505640] kobject: 'tx-26' (ffff880735cee718): kobject_uevent_env
[  156.505752] kobject: 'tx-26' (ffff880735cee718): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-26'
[  156.505885] kobject: 'tx-26' (ffff880735cee718): auto cleanup kobject_del
[  156.506014] kobject: 'tx-26' (ffff880735cee718): calling ktype release
[  156.506021] kobject: 'tx-26': free name
[  156.506175] kobject: 'tx-25' (ffff880735cee598): kobject_cleanup, parent ffff88075401a048
[  156.506181] kobject: 'tx-25' (ffff880735cee598): auto cleanup 'remove' event
[  156.506188] kobject: 'tx-25' (ffff880735cee598): kobject_uevent_env
[  156.506288] kobject: 'tx-25' (ffff880735cee598): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-25'
[  156.510576] kobject: 'tx-25' (ffff880735cee598): auto cleanup kobject_del
[  156.510753] kobject: 'tx-25' (ffff880735cee598): calling ktype release
[  156.510780] kobject: 'tx-25': free name
[  156.511032] kobject: 'tx-24' (ffff880735cee418): kobject_cleanup, parent ffff88075401a048
[  156.511040] kobject: 'tx-24' (ffff880735cee418): auto cleanup 'remove' event
[  156.511047] kobject: 'tx-24' (ffff880735cee418): kobject_uevent_env
[  156.511156] kobject: 'tx-24' (ffff880735cee418): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-24'
[  156.511335] kobject: 'tx-24' (ffff880735cee418): auto cleanup kobject_del
[  156.511518] kobject: 'tx-24' (ffff880735cee418): calling ktype release
[  156.511526] kobject: 'tx-24': free name
[  156.511711] kobject: 'tx-23' (ffff880735cee298): kobject_cleanup, parent ffff88075401a048
[  156.511719] kobject: 'tx-23' (ffff880735cee298): auto cleanup 'remove' event
[  156.511727] kobject: 'tx-23' (ffff880735cee298): kobject_uevent_env
[  156.511840] kobject: 'tx-23' (ffff880735cee298): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-23'
[  156.512017] kobject: 'tx-23' (ffff880735cee298): auto cleanup kobject_del
[  156.512198] kobject: 'tx-23' (ffff880735cee298): calling ktype release
[  156.512205] kobject: 'tx-23': free name
[  156.512359] kobject: 'tx-22' (ffff880735cee118): kobject_cleanup, parent ffff88075401a048
[  156.512367] kobject: 'tx-22' (ffff880735cee118): auto cleanup 'remove' event
[  156.512375] kobject: 'tx-22' (ffff880735cee118): kobject_uevent_env
[  156.512471] kobject: 'tx-22' (ffff880735cee118): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-22'
[  156.512666] kobject: 'tx-22' (ffff880735cee118): auto cleanup kobject_del
[  156.512802] kobject: 'tx-22' (ffff880735cee118): calling ktype release
[  156.512809] kobject: 'tx-22': free name
[  156.512953] kobject: 'tx-21' (ffff880735cedf98): kobject_cleanup, parent ffff88075401a048
[  156.512960] kobject: 'tx-21' (ffff880735cedf98): auto cleanup 'remove' event
[  156.512966] kobject: 'tx-21' (ffff880735cedf98): kobject_uevent_env
[  156.513050] kobject: 'tx-21' (ffff880735cedf98): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-21'
[  156.513174] kobject: 'tx-21' (ffff880735cedf98): auto cleanup kobject_del
[  156.513298] kobject: 'tx-21' (ffff880735cedf98): calling ktype release
[  156.513304] kobject: 'tx-21': free name
[  156.513438] kobject: 'tx-20' (ffff880735cede18): kobject_cleanup, parent ffff88075401a048
[  156.513445] kobject: 'tx-20' (ffff880735cede18): auto cleanup 'remove' event
[  156.513451] kobject: 'tx-20' (ffff880735cede18): kobject_uevent_env
[  156.513540] kobject: 'tx-20' (ffff880735cede18): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-20'
[  156.513682] kobject: 'tx-20' (ffff880735cede18): auto cleanup kobject_del
[  156.513809] kobject: 'tx-20' (ffff880735cede18): calling ktype release
[  156.513816] kobject: 'tx-20': free name
[  156.513957] kobject: 'tx-19' (ffff880735cedc98): kobject_cleanup, parent ffff88075401a048
[  156.513963] kobject: 'tx-19' (ffff880735cedc98): auto cleanup 'remove' event
[  156.513970] kobject: 'tx-19' (ffff880735cedc98): kobject_uevent_env
[  156.514058] kobject: 'tx-19' (ffff880735cedc98): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-19'
[  156.514168] kobject: 'tx-19' (ffff880735cedc98): auto cleanup kobject_del
[  156.514301] kobject: 'tx-19' (ffff880735cedc98): calling ktype release
[  156.514308] kobject: 'tx-19': free name
[  156.514551] kobject: 'tx-18' (ffff880735cedb18): kobject_cleanup, parent ffff88075401a048
[  156.514559] kobject: 'tx-18' (ffff880735cedb18): auto cleanup 'remove' event
[  156.514572] kobject: 'tx-18' (ffff880735cedb18): kobject_uevent_env
[  156.514708] kobject: 'tx-18' (ffff880735cedb18): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-18'
[  156.514938] kobject: 'tx-18' (ffff880735cedb18): auto cleanup kobject_del
[  156.515073] kobject: 'tx-18' (ffff880735cedb18): calling ktype release
[  156.515080] kobject: 'tx-18': free name
[  156.515264] kobject: 'tx-17' (ffff880735ced998): kobject_cleanup, parent ffff88075401a048
[  156.515270] kobject: 'tx-17' (ffff880735ced998): auto cleanup 'remove' event
[  156.515277] kobject: 'tx-17' (ffff880735ced998): kobject_uevent_env
[  156.515360] kobject: 'tx-17' (ffff880735ced998): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-17'
[  156.515497] kobject: 'tx-17' (ffff880735ced998): auto cleanup kobject_del
[  156.515619] kobject: 'tx-17' (ffff880735ced998): calling ktype release
[  156.515625] kobject: 'tx-17': free name
[  156.515772] kobject: 'tx-16' (ffff880735ced818): kobject_cleanup, parent ffff88075401a048
[  156.515778] kobject: 'tx-16' (ffff880735ced818): auto cleanup 'remove' event
[  156.515785] kobject: 'tx-16' (ffff880735ced818): kobject_uevent_env
[  156.515869] kobject: 'tx-16' (ffff880735ced818): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-16'
[  156.515981] kobject: 'tx-16' (ffff880735ced818): auto cleanup kobject_del
[  156.516084] kobject: 'tx-16' (ffff880735ced818): calling ktype release
[  156.516090] kobject: 'tx-16': free name
[  156.516296] kobject: 'tx-15' (ffff880735ced698): kobject_cleanup, parent ffff88075401a048
[  156.516302] kobject: 'tx-15' (ffff880735ced698): auto cleanup 'remove' event
[  156.516309] kobject: 'tx-15' (ffff880735ced698): kobject_uevent_env
[  156.516408] kobject: 'tx-15' (ffff880735ced698): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-15'
[  156.516536] kobject: 'tx-15' (ffff880735ced698): auto cleanup kobject_del
[  156.516675] kobject: 'tx-15' (ffff880735ced698): calling ktype release
[  156.516681] kobject: 'tx-15': free name
[  156.516836] kobject: 'tx-14' (ffff880735ced518): kobject_cleanup, parent ffff88075401a048
[  156.516843] kobject: 'tx-14' (ffff880735ced518): auto cleanup 'remove' event
[  156.516849] kobject: 'tx-14' (ffff880735ced518): kobject_uevent_env
[  156.516942] kobject: 'tx-14' (ffff880735ced518): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-14'
[  156.517079] kobject: 'tx-14' (ffff880735ced518): auto cleanup kobject_del
[  156.517209] kobject: 'tx-14' (ffff880735ced518): calling ktype release
[  156.517215] kobject: 'tx-14': free name
[  156.517354] kobject: 'tx-13' (ffff880735ced398): kobject_cleanup, parent ffff88075401a048
[  156.517361] kobject: 'tx-13' (ffff880735ced398): auto cleanup 'remove' event
[  156.517367] kobject: 'tx-13' (ffff880735ced398): kobject_uevent_env
[  156.517465] kobject: 'tx-13' (ffff880735ced398): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-13'
[  156.517588] kobject: 'tx-13' (ffff880735ced398): auto cleanup kobject_del
[  156.517707] kobject: 'tx-13' (ffff880735ced398): calling ktype release
[  156.517713] kobject: 'tx-13': free name
[  156.517872] kobject: 'tx-12' (ffff880735ced218): kobject_cleanup, parent ffff88075401a048
[  156.517879] kobject: 'tx-12' (ffff880735ced218): auto cleanup 'remove' event
[  156.517885] kobject: 'tx-12' (ffff880735ced218): kobject_uevent_env
[  156.517968] kobject: 'tx-12' (ffff880735ced218): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-12'
[  156.518089] kobject: 'tx-12' (ffff880735ced218): auto cleanup kobject_del
[  156.518215] kobject: 'tx-12' (ffff880735ced218): calling ktype release
[  156.518222] kobject: 'tx-12': free name
[  156.518363] kobject: 'tx-11' (ffff880735ced098): kobject_cleanup, parent ffff88075401a048
[  156.518369] kobject: 'tx-11' (ffff880735ced098): auto cleanup 'remove' event
[  156.518375] kobject: 'tx-11' (ffff880735ced098): kobject_uevent_env
[  156.518662] kobject: 'tx-11' (ffff880735ced098): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-11'
[  156.518915] kobject: 'tx-11' (ffff880735ced098): auto cleanup kobject_del
[  156.519079] kobject: 'tx-11' (ffff880735ced098): calling ktype release
[  156.519086] kobject: 'tx-11': free name
[  156.519263] kobject: 'tx-10' (ffff880735cecf18): kobject_cleanup, parent ffff88075401a048
[  156.519270] kobject: 'tx-10' (ffff880735cecf18): auto cleanup 'remove' event
[  156.519277] kobject: 'tx-10' (ffff880735cecf18): kobject_uevent_env
[  156.519371] kobject: 'tx-10' (ffff880735cecf18): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-10'
[  156.519515] kobject: 'tx-10' (ffff880735cecf18): auto cleanup kobject_del
[  156.519669] kobject: 'tx-10' (ffff880735cecf18): calling ktype release
[  156.519675] kobject: 'tx-10': free name
[  156.519827] kobject: 'tx-9' (ffff880735cecd98): kobject_cleanup, parent ffff88075401a048
[  156.519833] kobject: 'tx-9' (ffff880735cecd98): auto cleanup 'remove' event
[  156.519840] kobject: 'tx-9' (ffff880735cecd98): kobject_uevent_env
[  156.519937] kobject: 'tx-9' (ffff880735cecd98): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-9'
[  156.520064] kobject: 'tx-9' (ffff880735cecd98): auto cleanup kobject_del
[  156.520197] kobject: 'tx-9' (ffff880735cecd98): calling ktype release
[  156.520204] kobject: 'tx-9': free name
[  156.520329] kobject: 'tx-8' (ffff880735cecc18): kobject_cleanup, parent ffff88075401a048
[  156.520335] kobject: 'tx-8' (ffff880735cecc18): auto cleanup 'remove' event
[  156.520342] kobject: 'tx-8' (ffff880735cecc18): kobject_uevent_env
[  156.520414] kobject: 'tx-8' (ffff880735cecc18): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-8'
[  156.520567] kobject: 'tx-8' (ffff880735cecc18): auto cleanup kobject_del
[  156.520697] kobject: 'tx-8' (ffff880735cecc18): calling ktype release
[  156.520704] kobject: 'tx-8': free name
[  156.520845] kobject: 'tx-7' (ffff880735ceca98): kobject_cleanup, parent ffff88075401a048
[  156.520852] kobject: 'tx-7' (ffff880735ceca98): auto cleanup 'remove' event
[  156.520859] kobject: 'tx-7' (ffff880735ceca98): kobject_uevent_env
[  156.520937] kobject: 'tx-7' (ffff880735ceca98): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-7'
[  156.521066] kobject: 'tx-7' (ffff880735ceca98): auto cleanup kobject_del
[  156.521193] kobject: 'tx-7' (ffff880735ceca98): calling ktype release
[  156.521199] kobject: 'tx-7': free name
[  156.521350] kobject: 'tx-6' (ffff880735cec918): kobject_cleanup, parent ffff88075401a048
[  156.521357] kobject: 'tx-6' (ffff880735cec918): auto cleanup 'remove' event
[  156.521364] kobject: 'tx-6' (ffff880735cec918): kobject_uevent_env
[  156.521462] kobject: 'tx-6' (ffff880735cec918): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-6'
[  156.521602] kobject: 'tx-6' (ffff880735cec918): auto cleanup kobject_del
[  156.521736] kobject: 'tx-6' (ffff880735cec918): calling ktype release
[  156.521743] kobject: 'tx-6': free name
[  156.521899] kobject: 'tx-5' (ffff880735cec798): kobject_cleanup, parent ffff88075401a048
[  156.521905] kobject: 'tx-5' (ffff880735cec798): auto cleanup 'remove' event
[  156.521912] kobject: 'tx-5' (ffff880735cec798): kobject_uevent_env
[  156.522009] kobject: 'tx-5' (ffff880735cec798): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-5'
[  156.522143] kobject: 'tx-5' (ffff880735cec798): auto cleanup kobject_del
[  156.522287] kobject: 'tx-5' (ffff880735cec798): calling ktype release
[  156.522294] kobject: 'tx-5': free name
[  156.522538] kobject: 'tx-4' (ffff880735cec618): kobject_cleanup, parent ffff88075401a048
[  156.522547] kobject: 'tx-4' (ffff880735cec618): auto cleanup 'remove' event
[  156.522555] kobject: 'tx-4' (ffff880735cec618): kobject_uevent_env
[  156.522691] kobject: 'tx-4' (ffff880735cec618): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-4'
[  156.522960] kobject: 'tx-4' (ffff880735cec618): auto cleanup kobject_del
[  156.523100] kobject: 'tx-4' (ffff880735cec618): calling ktype release
[  156.523107] kobject: 'tx-4': free name
[  156.523323] kobject: 'tx-3' (ffff880735cec498): kobject_cleanup, parent ffff88075401a048
[  156.523337] kobject: 'tx-3' (ffff880735cec498): auto cleanup 'remove' event
[  156.523344] kobject: 'tx-3' (ffff880735cec498): kobject_uevent_env
[  156.523441] kobject: 'tx-3' (ffff880735cec498): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-3'
[  156.523585] kobject: 'tx-3' (ffff880735cec498): auto cleanup kobject_del
[  156.523770] kobject: 'tx-3' (ffff880735cec498): calling ktype release
[  156.523776] kobject: 'tx-3': free name
[  156.524003] kobject: 'tx-2' (ffff880735cec318): kobject_cleanup, parent ffff88075401a048
[  156.524009] kobject: 'tx-2' (ffff880735cec318): auto cleanup 'remove' event
[  156.524016] kobject: 'tx-2' (ffff880735cec318): kobject_uevent_env
[  156.524137] kobject: 'tx-2' (ffff880735cec318): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-2'
[  156.524279] kobject: 'tx-2' (ffff880735cec318): auto cleanup kobject_del
[  156.524413] kobject: 'tx-2' (ffff880735cec318): calling ktype release
[  156.524419] kobject: 'tx-2': free name
[  156.524605] kobject: 'tx-1' (ffff880735cec198): kobject_cleanup, parent ffff88075401a048
[  156.524612] kobject: 'tx-1' (ffff880735cec198): auto cleanup 'remove' event
[  156.524619] kobject: 'tx-1' (ffff880735cec198): kobject_uevent_env
[  156.524745] kobject: 'tx-1' (ffff880735cec198): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-1'
[  156.524915] kobject: 'tx-1' (ffff880735cec198): auto cleanup kobject_del
[  156.525094] kobject: 'tx-1' (ffff880735cec198): calling ktype release
[  156.525101] kobject: 'tx-1': free name
[  156.525294] kobject: 'tx-0' (ffff880735cec018): kobject_cleanup, parent ffff88075401a048
[  156.525301] kobject: 'tx-0' (ffff880735cec018): auto cleanup 'remove' event
[  156.525307] kobject: 'tx-0' (ffff880735cec018): kobject_uevent_env
[  156.525448] kobject: 'tx-0' (ffff880735cec018): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1/queues/tx-0'
[  156.525621] kobject: 'tx-0' (ffff880735cec018): auto cleanup kobject_del
[  156.525766] kobject: 'tx-0' (ffff880735cec018): calling ktype release
[  156.525773] kobject: 'tx-0': free name
[  156.525802] kobject: 'queues' (ffff88075401a048): kobject_cleanup, parent           (null)
[  156.525809] kobject: 'queues' (ffff88075401a048): calling ktype release
[  156.525816] kobject: 'queues' (ffff88075401a048): kset_release
[  156.525835] kobject: 'queues': free name
[  156.531193] PM: Removing info for No Bus:p4p1
[  156.531233] kobject: 'p4p1' (ffff88072fe005e0): kobject_uevent_env
[  156.531360] kobject: 'p4p1' (ffff88072fe005e0): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1'
[  156.531548] kobject: 'net' (ffff8807373a81c8): kobject_cleanup, parent ffff880751b480b8
[  156.531574] kobject: 'net' (ffff8807373a81c8): auto cleanup kobject_del
[  156.531611] kobject: 'net' (ffff8807373a81c8): calling ktype release
[  156.531660] kobject: 'net': free name
[  156.579530] kobject: 'p4p1' (ffff88072fe005e0): kobject_cleanup, parent           (null)
[  156.579551] kobject: 'p4p1' (ffff88072fe005e0): calling ktype release
[  156.579630] kobject: 'p4p1': free name
[  156.618740] kobject: '44' (ffff88075e452248): kobject_cleanup, parent           (null)
[  156.618751] kobject: '44' (ffff88075e452248): calling ktype release
[  156.619223] kobject: '44': free name
[  156.619306] kobject: '45' (ffff88075e454288): kobject_cleanup, parent           (null)
[  156.619313] kobject: '45' (ffff88075e454288): calling ktype release
[  156.619822] kobject: '45': free name
[  156.619940] kobject: '46' (ffff88075eb99788): kobject_cleanup, parent           (null)
[  156.619947] kobject: '46' (ffff88075eb99788): calling ktype release
[  156.620490] kobject: '46': free name
[  156.620594] kobject: '47' (ffff88075eb9d808): kobject_cleanup, parent           (null)
[  156.620600] kobject: '47' (ffff88075eb9d808): calling ktype release
[  156.621219] kobject: '47': free name
[  156.628866] kobject: '48' (ffff88075eb98768): kobject_cleanup, parent           (null)
[  156.628875] kobject: '48' (ffff88075eb98768): calling ktype release
[  156.629292] kobject: '48': free name
[  156.629372] kobject: '49' (ffff88075eb99228): kobject_cleanup, parent           (null)
[  156.629378] kobject: '49' (ffff88075eb99228): calling ktype release
[  156.629723] kobject: '49': free name
[  156.629786] kobject: '50' (ffff88075eb9b268): kobject_cleanup, parent           (null)
[  156.629792] kobject: '50' (ffff88075eb9b268): calling ktype release
[  156.630161] kobject: '50': free name
[  156.630242] kobject: '51' (ffff88075eb98208): kobject_cleanup, parent           (null)
[  156.630248] kobject: '51' (ffff88075eb98208): calling ktype release
[  156.630737] kobject: '51': free name
[  156.630912] kobject: '52' (ffff88075eb9a248): kobject_cleanup, parent           (null)
[  156.630920] kobject: '52' (ffff88075eb9a248): calling ktype release
[  156.631371] kobject: '52': free name
[  156.631511] kobject: '53' (ffff88075eb99ce8): kobject_cleanup, parent           (null)
[  156.631518] kobject: '53' (ffff88075eb99ce8): calling ktype release
[  156.631929] kobject: '53': free name
[  156.632002] kobject: '54' (ffff88075eb9e2c8): kobject_cleanup, parent           (null)
[  156.632008] kobject: '54' (ffff88075eb9e2c8): calling ktype release
[  156.632360] kobject: '54': free name
[  156.632441] kobject: '55' (ffff88075eb9ed88): kobject_cleanup, parent           (null)
[  156.632448] kobject: '55' (ffff88075eb9ed88): calling ktype release
[  156.632800] kobject: '55': free name
[  156.632875] kobject: '56' (ffff88075eb9bd28): kobject_cleanup, parent           (null)
[  156.632881] kobject: '56' (ffff88075eb9bd28): calling ktype release
[  156.633263] kobject: '56': free name
[  156.633362] kobject: '57' (ffff88075eb9b7c8): kobject_cleanup, parent           (null)
[  156.633368] kobject: '57' (ffff88075eb9b7c8): calling ktype release
[  156.633729] kobject: '57': free name
[  156.633800] kobject: '58' (ffff88075eb9a7a8): kobject_cleanup, parent           (null)
[  156.633807] kobject: '58' (ffff88075eb9a7a8): calling ktype release
[  156.634166] kobject: '58': free name
[  156.634232] kobject: '59' (ffff88075eb98cc8): kobject_cleanup, parent           (null)
[  156.634238] kobject: '59' (ffff88075eb98cc8): calling ktype release
[  156.634687] kobject: '59': free name
[  156.634800] kobject: '60' (ffff88075eb9e828): kobject_cleanup, parent           (null)
[  156.634808] kobject: '60' (ffff88075eb9e828): calling ktype release
[  156.635308] kobject: '60': free name
[  156.635408] kobject: '61' (ffff88075eb9d2a8): kobject_cleanup, parent           (null)
[  156.635414] kobject: '61' (ffff88075eb9d2a8): calling ktype release
[  156.635796] kobject: '61': free name
[  156.635868] kobject: '62' (ffff88075eb9ad08): kobject_cleanup, parent           (null)
[  156.635874] kobject: '62' (ffff88075eb9ad08): calling ktype release
[  156.636220] kobject: '62': free name
[  156.636284] kobject: '63' (ffff88075eb9c288): kobject_cleanup, parent           (null)
[  156.636290] kobject: '63' (ffff88075eb9c288): calling ktype release
[  156.636629] kobject: '63': free name
[  156.636697] kobject: '64' (ffff88075eb9c7e8): kobject_cleanup, parent           (null)
[  156.636703] kobject: '64' (ffff88075eb9c7e8): calling ktype release
[  156.637027] kobject: '64': free name
[  156.637161] kobject: '65' (ffff88075eb9cd48): kobject_cleanup, parent           (null)
[  156.637167] kobject: '65' (ffff88075eb9cd48): calling ktype release
[  156.637517] kobject: '65': free name
[  159.728111] nfp 0000:82:00.0: Firmware safely unloaded
[  159.734968] device: 'cpp': device_unregister
[  159.735303] PM: Removing info for No Bus:cpp
[  159.735324] kobject: 'cpp' (ffff8807320b0978): kobject_uevent_env
[  159.735332] kobject: 'cpp' (ffff8807320b0978): kobject_uevent_env: filter function caused the event to drop!
[  159.735387] kobject: 'cpp' (ffff8807320b0978): kobject_cleanup, parent           (null)
[  159.735395] kobject: 'cpp' (ffff8807320b0978): calling ktype release
[  159.735421] kobject: 'cpp': free name
[  159.750983] firmware_class: fw_name_devm_release: fw_name-netronome/nic_AMDA0081-0001_1x40.nffw devm-ffff88074a6c1060 released
[  159.752467] kobject: '0000:82:00.0' (ffff880751b480b8): kobject_uevent_env
[  159.752601] kobject: '0000:82:00.0' (ffff880751b480b8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0'
[  159.752949] kobject: 'nfp' (ffff880738121a28): kobject_cleanup, parent ffff88045331bbe8
[  159.752957] kobject: 'nfp' (ffff880738121a28): auto cleanup 'remove' event
[  159.752964] kobject: 'nfp' (ffff880738121a28): kobject_uevent_env
[  159.753052] kobject: 'nfp' (ffff880738121a28): fill_kobj_path: path = '/bus/pci/drivers/nfp'
[  159.753201] kobject: 'nfp' (ffff880738121a28): auto cleanup kobject_del
[  159.753267] kobject: 'nfp' (ffff880738121a28): calling ktype release
[  159.753274] driver: 'nfp': driver_release
[  159.753299] kobject: 'nfp': free name
[  159.753953] kobject: 'drivers' (ffff880736b92a48): kobject_cleanup, parent ffffffffc02dc6d0
[  159.753961] kobject: 'drivers' (ffff880736b92a48): auto cleanup kobject_del
[  159.753983] kobject: 'drivers' (ffff880736b92a48): calling ktype release
[  159.753990] kobject: (ffff880736b92a48): dynamic_kobj_release
[  159.754010] kobject: 'drivers': free name
[  159.754017] kobject: 'holders' (ffff880738539528): kobject_cleanup, parent ffffffffc02dc6d0
[  159.754024] kobject: 'holders' (ffff880738539528): auto cleanup kobject_del
[  159.754035] kobject: 'holders' (ffff880738539528): calling ktype release
[  159.754042] kobject: (ffff880738539528): dynamic_kobj_release
[  159.754074] kobject: 'holders': free name
[  159.754137] kobject: 'notes' (ffff880738536d88): kobject_cleanup, parent ffffffffc02dc6d0
[  159.754144] kobject: 'notes' (ffff880738536d88): auto cleanup kobject_del
[  159.754167] kobject: 'notes' (ffff880738536d88): calling ktype release
[  159.754174] kobject: (ffff880738536d88): dynamic_kobj_release
[  159.754197] kobject: 'notes': free name
[  159.756358] kobject: 'nfp' (ffffffffc02dc6d0): kobject_cleanup, parent ffff880452c706c8
[  159.756366] kobject: 'nfp' (ffffffffc02dc6d0): auto cleanup 'remove' event
[  159.756373] kobject: 'nfp' (ffffffffc02dc6d0): kobject_uevent_env
[  159.756511] kobject: 'nfp' (ffffffffc02dc6d0): fill_kobj_path: path = '/module/nfp'
[  159.756706] kobject: 'nfp' (ffffffffc02dc6d0): auto cleanup kobject_del
[  159.756720] kobject: 'nfp' (ffffffffc02dc6d0): calling ktype release
[  159.756729] kobject: 'nfp': free name
[  159.772475] kobject: '43' (ffff88074c7ca7a8): kobject_cleanup, parent           (null)
[  159.772484] kobject: '43' (ffff88074c7ca7a8): calling ktype release
[  159.773162] kobject: '43': free name
[  166.793852] kobject: 'nfp' (ffffffffc10ec6d0): kobject_add_internal: parent: 'module', set: 'module'
[  166.793984] kobject: 'holders' (ffff880469bb9ba8): kobject_add_internal: parent: 'nfp', set: '<NULL>'
[  166.795913] kobject: 'notes' (ffff880469bb81a8): kobject_add_internal: parent: 'nfp', set: '<NULL>'
[  166.796042] kobject: 'nfp' (ffffffffc10ec6d0): kobject_uevent_env
[  166.796104] kobject: 'nfp' (ffffffffc10ec6d0): fill_kobj_path: path = '/module/nfp'
[  166.802192] nfp: NFP PCIe Driver, Copyright (C) 2014-2017 Netronome Systems
[  166.810203] bus: 'pci': add driver nfp
[  166.810276] kobject: 'nfp' (ffff880468d650a8): kobject_add_internal: parent: 'drivers', set: 'drivers'
[  166.810413] bus: 'pci': driver_probe_device: matched device 0000:04:00.0 with driver nfp
[  166.810428] bus: 'pci': really_probe: probing driver nfp with device 0000:04:00.0
[  166.810526] nfp 0000:04:00.0: no default pinctrl state
[  166.810667] devices_kset: Moving 0000:04:00.0 to end of list
[  166.810677] nfp 0000:04:00.0: runtime IRQ mapping not provided by arch
[  166.813648] kobject: '43' (ffff880446065808): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  166.814131] nfp 0000:04:00.0: enabling bus mastering
[  166.814902] nfp 0000:04:00.0: Netronome Flow Processor NFP4000/NFP6000 PCIe Card Probe
[  166.824558] nfp 0000:04:00.0: RESERVED BARs: 0.0: General/MSI-X SRAM, 0.1: PCIe XPB/MSI-X PBA, 0.4: Explicit0, 0.5: Explicit1, free: 20/24
[  166.838721] device: 'cpp': device_add
[  166.838731] kobject: 'cpp' (ffff880446c24b18): kobject_add_internal: parent: '0000:04:00.0', set: 'devices'
[  166.839115] PM: Adding info for No Bus:cpp
[  166.839126] kobject: 'cpp' (ffff880446c24b18): kobject_uevent_env
[  166.839133] kobject: 'cpp' (ffff880446c24b18): kobject_uevent_env: filter function caused the event to drop!
[  166.840893] nfp 0000:04:00.0: Model: 0x60010010, SN: 00:15:4d:0a:0c:23, Ifc: 0x10ff
[  166.857625] nfp 0000:04:00.0: Assembly: SMAAMDA0058-000215171419-01 CPLD: 0x44
[  167.115900] nfp 0000:04:00.0: BSP: 020014.020014.020049
[  167.121879] nfp 0000:04:00.0: nfp: Looking for firmware file in order of priority:
[  167.130565] firmware_class: __allocate_fw_buf: fw-netronome/serial-00-15-4d-0a-0c-23-10-ff.nffw buf=ffff88045f779448
[  167.131029] nfp 0000:04:00.0: loading /lib/firmware/updates/4.13.0-debug-05272-g9beb8bedb05c/netronome/serial-00-15-4d-0a-0c-23-10-ff.nffw failed with error -2
[  167.131130] nfp 0000:04:00.0: loading /lib/firmware/updates/netronome/serial-00-15-4d-0a-0c-23-10-ff.nffw failed with error -2
[  167.131318] nfp 0000:04:00.0: loading /lib/firmware/4.13.0-debug-05272-g9beb8bedb05c/netronome/serial-00-15-4d-0a-0c-23-10-ff.nffw failed with error -2
[  167.147608] nfp 0000:04:00.0: loading /lib/firmware/netronome/serial-00-15-4d-0a-0c-23-10-ff.nffw failed with error -2
[  167.147631] firmware_class: __fw_free_buf: fw-netronome/serial-00-15-4d-0a-0c-23-10-ff.nffw buf=ffff88045f779448 data=          (null) size=0
[  167.147662] nfp 0000:04:00.0: nfp:   netronome/serial-00-15-4d-0a-0c-23-10-ff.nffw: not found
[  167.157554] firmware_class: __allocate_fw_buf: fw-netronome/pci-0000:04:00.0.nffw buf=ffff88045f77ca48
[  167.157744] nfp 0000:04:00.0: loading /lib/firmware/updates/4.13.0-debug-05272-g9beb8bedb05c/netronome/pci-0000:04:00.0.nffw failed with error -2
[  167.157874] nfp 0000:04:00.0: loading /lib/firmware/updates/netronome/pci-0000:04:00.0.nffw failed with error -2
[  167.157986] nfp 0000:04:00.0: loading /lib/firmware/4.13.0-debug-05272-g9beb8bedb05c/netronome/pci-0000:04:00.0.nffw failed with error -2
[  167.158145] nfp 0000:04:00.0: loading /lib/firmware/netronome/pci-0000:04:00.0.nffw failed with error -2
[  167.158163] firmware_class: __fw_free_buf: fw-netronome/pci-0000:04:00.0.nffw buf=ffff88045f77ca48 data=          (null) size=0
[  167.158194] nfp 0000:04:00.0: nfp:   netronome/pci-0000:04:00.0.nffw: not found
[  167.166588] firmware_class: __allocate_fw_buf: fw-netronome/nic_AMDA0058-0002_2x40.nffw buf=ffff88045f77abe8
[  167.166767] nfp 0000:04:00.0: loading /lib/firmware/updates/4.13.0-debug-05272-g9beb8bedb05c/netronome/nic_AMDA0058-0002_2x40.nffw failed with error -2
[  167.166881] nfp 0000:04:00.0: loading /lib/firmware/updates/netronome/nic_AMDA0058-0002_2x40.nffw failed with error -2
[  167.167000] nfp 0000:04:00.0: loading /lib/firmware/4.13.0-debug-05272-g9beb8bedb05c/netronome/nic_AMDA0058-0002_2x40.nffw failed with error -2
[  167.167150] nfp 0000:04:00.0: loading /lib/firmware/netronome/nic_AMDA0058-0002_2x40.nffw failed with error -2
[  167.167167] firmware_class: __fw_free_buf: fw-netronome/nic_AMDA0058-0002_2x40.nffw buf=ffff88045f77abe8 data=          (null) size=0
[  167.167199] nfp 0000:04:00.0: nfp:   netronome/nic_AMDA0058-0002_2x40.nffw: not found
[  167.177971] nfp 0000:04:00.0: nfp: No firmware found, giving up.
[  167.186228] device: 'cpp': device_unregister
[  167.186515] PM: Removing info for No Bus:cpp
[  167.186530] kobject: 'cpp' (ffff880446c24b18): kobject_uevent_env
[  167.186537] kobject: 'cpp' (ffff880446c24b18): kobject_uevent_env: filter function caused the event to drop!
[  167.186574] kobject: 'cpp' (ffff880446c24b18): kobject_cleanup, parent           (null)
[  167.186581] kobject: 'cpp' (ffff880446c24b18): calling ktype release
[  167.186599] kobject: 'cpp': free name
[  167.189522] nfp: probe of 0000:04:00.0 failed with error -22
[  167.196050] bus: 'pci': driver_probe_device: matched device 0000:05:00.0 with driver nfp
[  167.196063] bus: 'pci': really_probe: probing driver nfp with device 0000:05:00.0
[  167.196176] nfp 0000:05:00.0: no default pinctrl state
[  167.196318] devices_kset: Moving 0000:05:00.0 to end of list
[  167.196327] nfp 0000:05:00.0: runtime IRQ mapping not provided by arch
[  167.198915] kobject: '43' (ffff88045d740cc8): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  167.199382] nfp 0000:05:00.0: enabling bus mastering
[  167.200392] nfp 0000:05:00.0: Netronome Flow Processor NFP4000/NFP6000 PCIe Card Probe
[  167.209944] nfp 0000:05:00.0: RESERVED BARs: 0.0: General/MSI-X SRAM, 0.1: PCIe XPB/MSI-X PBA, 0.4: Explicit0, 0.5: Explicit1, free: 20/24
[  167.224451] device: 'cpp': device_add
[  167.224460] kobject: 'cpp' (ffff880446c21c38): kobject_add_internal: parent: '0000:05:00.0', set: 'devices'
[  167.224814] PM: Adding info for No Bus:cpp
[  167.224824] kobject: 'cpp' (ffff880446c21c38): kobject_uevent_env
[  167.224831] kobject: 'cpp' (ffff880446c21c38): kobject_uevent_env: filter function caused the event to drop!
[  167.225698] nfp 0000:05:00.0: Model: 0x40010010, SN: 00:15:4d:12:20:7e, Ifc: 0x10ff
[  167.236293] kobject: '43' (ffff880446065808): kobject_cleanup, parent           (null)
[  167.236302] kobject: '43' (ffff880446065808): calling ktype release
[  167.236454] kobject: '43': free name
[  167.243196] nfp 0000:05:00.0: Assembly: SMAAMDA0099-000116500380-07 CPLD: 0x3010000
[  167.504125] nfp 0000:05:00.0: BSP: 020014.020014.020049
[  167.510123] nfp 0000:05:00.0: nfp: Looking for firmware file in order of priority:
[  167.518832] firmware_class: __allocate_fw_buf: fw-netronome/serial-00-15-4d-12-20-7e-10-ff.nffw buf=ffff880451595e88
[  167.519002] nfp 0000:05:00.0: loading /lib/firmware/updates/4.13.0-debug-05272-g9beb8bedb05c/netronome/serial-00-15-4d-12-20-7e-10-ff.nffw failed with error -2
[  167.519117] nfp 0000:05:00.0: loading /lib/firmware/updates/netronome/serial-00-15-4d-12-20-7e-10-ff.nffw failed with error -2
[  167.519244] nfp 0000:05:00.0: loading /lib/firmware/4.13.0-debug-05272-g9beb8bedb05c/netronome/serial-00-15-4d-12-20-7e-10-ff.nffw failed with error -2
[  167.519430] nfp 0000:05:00.0: loading /lib/firmware/netronome/serial-00-15-4d-12-20-7e-10-ff.nffw failed with error -2
[  167.519447] firmware_class: __fw_free_buf: fw-netronome/serial-00-15-4d-12-20-7e-10-ff.nffw buf=ffff880451595e88 data=          (null) size=0
[  167.519477] nfp 0000:05:00.0: nfp:   netronome/serial-00-15-4d-12-20-7e-10-ff.nffw: not found
[  167.529289] firmware_class: __allocate_fw_buf: fw-netronome/pci-0000:05:00.0.nffw buf=ffff880451591448
[  167.529462] nfp 0000:05:00.0: loading /lib/firmware/updates/4.13.0-debug-05272-g9beb8bedb05c/netronome/pci-0000:05:00.0.nffw failed with error -2
[  167.529595] nfp 0000:05:00.0: loading /lib/firmware/updates/netronome/pci-0000:05:00.0.nffw failed with error -2
[  167.529721] nfp 0000:05:00.0: loading /lib/firmware/4.13.0-debug-05272-g9beb8bedb05c/netronome/pci-0000:05:00.0.nffw failed with error -2
[  167.529873] nfp 0000:05:00.0: loading /lib/firmware/netronome/pci-0000:05:00.0.nffw failed with error -2
[  167.529889] firmware_class: __fw_free_buf: fw-netronome/pci-0000:05:00.0.nffw buf=ffff880451591448 data=          (null) size=0
[  167.529920] nfp 0000:05:00.0: nfp:   netronome/pci-0000:05:00.0.nffw: not found
[  167.538344] firmware_class: __allocate_fw_buf: fw-netronome/nic_AMDA0099-0001_2x25.nffw buf=ffff88046a1f90e8
[  167.538504] nfp 0000:05:00.0: loading /lib/firmware/updates/4.13.0-debug-05272-g9beb8bedb05c/netronome/nic_AMDA0099-0001_2x25.nffw failed with error -2
[  167.538617] nfp 0000:05:00.0: loading /lib/firmware/updates/netronome/nic_AMDA0099-0001_2x25.nffw failed with error -2
[  167.538744] nfp 0000:05:00.0: loading /lib/firmware/4.13.0-debug-05272-g9beb8bedb05c/netronome/nic_AMDA0099-0001_2x25.nffw failed with error -2
[  167.538893] nfp 0000:05:00.0: loading /lib/firmware/netronome/nic_AMDA0099-0001_2x25.nffw failed with error -2
[  167.538908] firmware_class: __fw_free_buf: fw-netronome/nic_AMDA0099-0001_2x25.nffw buf=ffff88046a1f90e8 data=          (null) size=0
[  167.538939] nfp 0000:05:00.0: nfp:   netronome/nic_AMDA0099-0001_2x25.nffw: not found
[  167.549694] nfp 0000:05:00.0: nfp: No firmware found, giving up.
[  167.557151] device: 'cpp': device_unregister
[  167.557407] PM: Removing info for No Bus:cpp
[  167.557421] kobject: 'cpp' (ffff880446c21c38): kobject_uevent_env
[  167.557428] kobject: 'cpp' (ffff880446c21c38): kobject_uevent_env: filter function caused the event to drop!
[  167.557458] kobject: 'cpp' (ffff880446c21c38): kobject_cleanup, parent           (null)
[  167.557464] kobject: 'cpp' (ffff880446c21c38): calling ktype release
[  167.557482] kobject: 'cpp': free name
[  167.560016] nfp: probe of 0000:05:00.0 failed with error -22
[  167.566699] bus: 'pci': driver_probe_device: matched device 0000:82:00.0 with driver nfp
[  167.566711] bus: 'pci': really_probe: probing driver nfp with device 0000:82:00.0
[  167.566813] nfp 0000:82:00.0: no default pinctrl state
[  167.566940] devices_kset: Moving 0000:82:00.0 to end of list
[  167.566949] nfp 0000:82:00.0: runtime IRQ mapping not provided by arch
[  167.569677] kobject: '43' (ffff880753ef0208): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  167.570166] nfp 0000:82:00.0: enabling bus mastering
[  167.570890] nfp 0000:82:00.0: Netronome Flow Processor NFP4000/NFP6000 PCIe Card Probe
[  167.588701] kobject: '43' (ffff88045d740cc8): kobject_cleanup, parent           (null)
[  167.588709] kobject: '43' (ffff88045d740cc8): calling ktype release
[  167.588865] kobject: '43': free name
[  167.590477] nfp 0000:82:00.0: RESERVED BARs: 0.0: General/MSI-X SRAM, 0.1: PCIe XPB/MSI-X PBA, 0.4: Explicit0, 0.5: Explicit1, free: 20/24
[  167.604668] device: 'cpp': device_add
[  167.604678] kobject: 'cpp' (ffff88073072ddd8): kobject_add_internal: parent: '0000:82:00.0', set: 'devices'
[  167.605074] PM: Adding info for No Bus:cpp
[  167.605087] kobject: 'cpp' (ffff88073072ddd8): kobject_uevent_env
[  167.605094] kobject: 'cpp' (ffff88073072ddd8): kobject_uevent_env: filter function caused the event to drop!
[  167.605995] nfp 0000:82:00.0: Model: 0x40010010, SN: 00:15:4d:12:27:6a, Ifc: 0x10ff
[  167.622891] nfp 0000:82:00.0: Assembly: SMAAMDA0081-000116240145-15 CPLD: 0x1030000
[  167.920242] nfp 0000:82:00.0: BSP: 020014.020014.020049
[  167.926256] nfp 0000:82:00.0: nfp: Looking for firmware file in order of priority:
[  167.934984] firmware_class: __allocate_fw_buf: fw-netronome/serial-00-15-4d-12-27-6a-10-ff.nffw buf=ffff8807370b5e88
[  167.935152] nfp 0000:82:00.0: loading /lib/firmware/updates/4.13.0-debug-05272-g9beb8bedb05c/netronome/serial-00-15-4d-12-27-6a-10-ff.nffw failed with error -2
[  167.935259] nfp 0000:82:00.0: loading /lib/firmware/updates/netronome/serial-00-15-4d-12-27-6a-10-ff.nffw failed with error -2
[  167.935355] nfp 0000:82:00.0: loading /lib/firmware/4.13.0-debug-05272-g9beb8bedb05c/netronome/serial-00-15-4d-12-27-6a-10-ff.nffw failed with error -2
[  167.935568] nfp 0000:82:00.0: loading /lib/firmware/netronome/serial-00-15-4d-12-27-6a-10-ff.nffw failed with error -2
[  167.935591] firmware_class: __fw_free_buf: fw-netronome/serial-00-15-4d-12-27-6a-10-ff.nffw buf=ffff8807370b5e88 data=          (null) size=0
[  167.935619] nfp 0000:82:00.0: nfp:   netronome/serial-00-15-4d-12-27-6a-10-ff.nffw: not found
[  167.945422] firmware_class: __allocate_fw_buf: fw-netronome/pci-0000:82:00.0.nffw buf=ffff8807370b32a8
[  167.945562] nfp 0000:82:00.0: loading /lib/firmware/updates/4.13.0-debug-05272-g9beb8bedb05c/netronome/pci-0000:82:00.0.nffw failed with error -2
[  167.945680] nfp 0000:82:00.0: loading /lib/firmware/updates/netronome/pci-0000:82:00.0.nffw failed with error -2
[  167.945785] nfp 0000:82:00.0: loading /lib/firmware/4.13.0-debug-05272-g9beb8bedb05c/netronome/pci-0000:82:00.0.nffw failed with error -2
[  167.945946] nfp 0000:82:00.0: loading /lib/firmware/netronome/pci-0000:82:00.0.nffw failed with error -2
[  167.945967] firmware_class: __fw_free_buf: fw-netronome/pci-0000:82:00.0.nffw buf=ffff8807370b32a8 data=          (null) size=0
[  167.946001] nfp 0000:82:00.0: nfp:   netronome/pci-0000:82:00.0.nffw: not found
[  167.954425] firmware_class: __allocate_fw_buf: fw-netronome/nic_AMDA0081-0001_1x40.nffw buf=ffff8807370b5468
[  167.954555] nfp 0000:82:00.0: loading /lib/firmware/updates/4.13.0-debug-05272-g9beb8bedb05c/netronome/nic_AMDA0081-0001_1x40.nffw failed with error -2
[  167.954661] nfp 0000:82:00.0: loading /lib/firmware/updates/netronome/nic_AMDA0081-0001_1x40.nffw failed with error -2
[  167.954812] nfp 0000:82:00.0: loading /lib/firmware/4.13.0-debug-05272-g9beb8bedb05c/netronome/nic_AMDA0081-0001_1x40.nffw failed with error -2
[  167.988895] nfp 0000:82:00.0: direct-loading netronome/nic_AMDA0081-0001_1x40.nffw
[  167.988998] firmware_class: fw_set_page_data: fw-netronome/nic_AMDA0081-0001_1x40.nffw buf=ffff8807370b5468 data=ffffc900053c1000 size=2385584
[  167.989008] nfp 0000:82:00.0: nfp:   netronome/nic_AMDA0081-0001_1x40.nffw: found, loading...
[  167.998852] nfp 0000:82:00.0: Soft-reset, loading FW image
[  181.338932] nfp 0000:82:00.0: Finished loading FW image
[  181.344950] firmware_class: __fw_free_buf: fw-netronome/nic_AMDA0081-0001_1x40.nffw buf=ffff8807370b5468 data=ffffc900053c1000 size=2385584
[  181.356838] kobject: '44' (ffff880753972d08): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.357706] kobject: '45' (ffff880753972248): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.358367] kobject: '46' (ffff880753974d48): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.359153] kobject: '47' (ffff880753975808): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.359843] kobject: '48' (ffff8807539752a8): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.360528] kobject: '49' (ffff880753971788): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.361191] kobject: '50' (ffff880753971ce8): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.361872] kobject: '51' (ffff880753970cc8): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.362551] kobject: '52' (ffff880753971228): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.363264] kobject: '53' (ffff880753974288): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.363925] kobject: '54' (ffff880754d9b268): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.364608] kobject: '55' (ffff88075e8f8cc8): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.365260] kobject: '56' (ffff88075e8f8208): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.365935] kobject: '57' (ffff88075e8fc7e8): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.366589] kobject: '58' (ffff88075e8f9788): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.367315] kobject: '59' (ffff88075e8fe828): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.367973] kobject: '60' (ffff88075e8fb268): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.368650] kobject: '61' (ffff88075e8fbd28): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.369331] kobject: '62' (ffff88075e8fe2c8): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.370007] kobject: '63' (ffff88075e8ff2e8): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.370659] kobject: '64' (ffff88075e8f9228): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.371471] kobject: '65' (ffff88075e8fed88): kobject_add_internal: parent: 'irq', set: '<NULL>'
[  181.383042] device: 'eth0': device_add
[  181.383150] kobject: 'net' (ffff880752014008): kobject_add_internal: parent: '0000:82:00.0', set: '(null)'
[  181.383266] kobject: 'eth0' (ffff8807333d05e0): kobject_add_internal: parent: 'net', set: 'devices'
[  181.385121] PM: Adding info for No Bus:eth0
[  181.385138] kobject: 'eth0' (ffff8807333d05e0): kobject_uevent_env
[  181.385204] kobject: 'eth0' (ffff8807333d05e0): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0'
[  181.385515] kobject: 'queues' (ffff88072f9504a8): kobject_add_internal: parent: 'eth0', set: '<NULL>'
[  181.385551] kobject: 'queues' (ffff88072f9504a8): kobject_uevent_env
[  181.385559] kobject: 'queues' (ffff88072f9504a8): kobject_uevent_env: filter function caused the event to drop!
[  181.385593] kobject: 'rx-0' (ffff880760c76858): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.385704] kobject: 'rx-0' (ffff880760c76858): kobject_uevent_env
[  181.385776] kobject: 'rx-0' (ffff880760c76858): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-0'
[  181.385903] kobject: 'rx-1' (ffff880760c768d8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.386012] kobject: 'rx-1' (ffff880760c768d8): kobject_uevent_env
[  181.386072] kobject: 'rx-1' (ffff880760c768d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-1'
[  181.386181] kobject: 'rx-2' (ffff880760c76958): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.386286] kobject: 'rx-2' (ffff880760c76958): kobject_uevent_env
[  181.386345] kobject: 'rx-2' (ffff880760c76958): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-2'
[  181.386451] kobject: 'rx-3' (ffff880760c769d8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.386554] kobject: 'rx-3' (ffff880760c769d8): kobject_uevent_env
[  181.386615] kobject: 'rx-3' (ffff880760c769d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-3'
[  181.386723] kobject: 'rx-4' (ffff880760c76a58): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.386888] kobject: 'rx-4' (ffff880760c76a58): kobject_uevent_env
[  181.386951] kobject: 'rx-4' (ffff880760c76a58): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-4'
[  181.387288] kobject: 'rx-5' (ffff880760c76ad8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.387410] kobject: 'rx-5' (ffff880760c76ad8): kobject_uevent_env
[  181.387475] kobject: 'rx-5' (ffff880760c76ad8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-5'
[  181.387592] kobject: 'rx-6' (ffff880760c76b58): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.387698] kobject: 'rx-6' (ffff880760c76b58): kobject_uevent_env
[  181.387762] kobject: 'rx-6' (ffff880760c76b58): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-6'
[  181.387873] kobject: 'rx-7' (ffff880760c76bd8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.387979] kobject: 'rx-7' (ffff880760c76bd8): kobject_uevent_env
[  181.388040] kobject: 'rx-7' (ffff880760c76bd8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-7'
[  181.388153] kobject: 'rx-8' (ffff880760c76c58): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.388274] kobject: 'rx-8' (ffff880760c76c58): kobject_uevent_env
[  181.388338] kobject: 'rx-8' (ffff880760c76c58): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-8'
[  181.388451] kobject: 'rx-9' (ffff880760c76cd8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.388560] kobject: 'rx-9' (ffff880760c76cd8): kobject_uevent_env
[  181.388619] kobject: 'rx-9' (ffff880760c76cd8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-9'
[  181.388730] kobject: 'rx-10' (ffff880760c76d58): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.388850] kobject: 'rx-10' (ffff880760c76d58): kobject_uevent_env
[  181.388920] kobject: 'rx-10' (ffff880760c76d58): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-10'
[  181.389037] kobject: 'rx-11' (ffff880760c76dd8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.389147] kobject: 'rx-11' (ffff880760c76dd8): kobject_uevent_env
[  181.389205] kobject: 'rx-11' (ffff880760c76dd8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-11'
[  181.389317] kobject: 'rx-12' (ffff880760c76e58): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.389423] kobject: 'rx-12' (ffff880760c76e58): kobject_uevent_env
[  181.389483] kobject: 'rx-12' (ffff880760c76e58): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-12'
[  181.389592] kobject: 'rx-13' (ffff880760c76ed8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.389700] kobject: 'rx-13' (ffff880760c76ed8): kobject_uevent_env
[  181.389759] kobject: 'rx-13' (ffff880760c76ed8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-13'
[  181.389871] kobject: 'rx-14' (ffff880760c76f58): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.389977] kobject: 'rx-14' (ffff880760c76f58): kobject_uevent_env
[  181.390038] kobject: 'rx-14' (ffff880760c76f58): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-14'
[  181.390149] kobject: 'rx-15' (ffff880760c76fd8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.390252] kobject: 'rx-15' (ffff880760c76fd8): kobject_uevent_env
[  181.390316] kobject: 'rx-15' (ffff880760c76fd8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-15'
[  181.390427] kobject: 'rx-16' (ffff880760c77058): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.390534] kobject: 'rx-16' (ffff880760c77058): kobject_uevent_env
[  181.390596] kobject: 'rx-16' (ffff880760c77058): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-16'
[  181.390710] kobject: 'rx-17' (ffff880760c770d8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.390866] kobject: 'rx-17' (ffff880760c770d8): kobject_uevent_env
[  181.390947] kobject: 'rx-17' (ffff880760c770d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-17'
[  181.391087] kobject: 'rx-18' (ffff880760c77158): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.391233] kobject: 'rx-18' (ffff880760c77158): kobject_uevent_env
[  181.391297] kobject: 'rx-18' (ffff880760c77158): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-18'
[  181.391441] kobject: 'rx-19' (ffff880760c771d8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.391554] kobject: 'rx-19' (ffff880760c771d8): kobject_uevent_env
[  181.391615] kobject: 'rx-19' (ffff880760c771d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-19'
[  181.391728] kobject: 'rx-20' (ffff880760c77258): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.391838] kobject: 'rx-20' (ffff880760c77258): kobject_uevent_env
[  181.391898] kobject: 'rx-20' (ffff880760c77258): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-20'
[  181.392010] kobject: 'rx-21' (ffff880760c772d8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.392119] kobject: 'rx-21' (ffff880760c772d8): kobject_uevent_env
[  181.392178] kobject: 'rx-21' (ffff880760c772d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-21'
[  181.392290] kobject: 'rx-22' (ffff880760c77358): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.392401] kobject: 'rx-22' (ffff880760c77358): kobject_uevent_env
[  181.392460] kobject: 'rx-22' (ffff880760c77358): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-22'
[  181.392573] kobject: 'rx-23' (ffff880760c773d8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.392681] kobject: 'rx-23' (ffff880760c773d8): kobject_uevent_env
[  181.392740] kobject: 'rx-23' (ffff880760c773d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-23'
[  181.392853] kobject: 'rx-24' (ffff880760c77458): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.392960] kobject: 'rx-24' (ffff880760c77458): kobject_uevent_env
[  181.393033] kobject: 'rx-24' (ffff880760c77458): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-24'
[  181.393151] kobject: 'rx-25' (ffff880760c774d8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.393259] kobject: 'rx-25' (ffff880760c774d8): kobject_uevent_env
[  181.393319] kobject: 'rx-25' (ffff880760c774d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-25'
[  181.393433] kobject: 'rx-26' (ffff880760c77558): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.393538] kobject: 'rx-26' (ffff880760c77558): kobject_uevent_env
[  181.393597] kobject: 'rx-26' (ffff880760c77558): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-26'
[  181.393708] kobject: 'rx-27' (ffff880760c775d8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.393819] kobject: 'rx-27' (ffff880760c775d8): kobject_uevent_env
[  181.393879] kobject: 'rx-27' (ffff880760c775d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-27'
[  181.393991] kobject: 'rx-28' (ffff880760c77658): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.394113] kobject: 'rx-28' (ffff880760c77658): kobject_uevent_env
[  181.394173] kobject: 'rx-28' (ffff880760c77658): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-28'
[  181.394288] kobject: 'rx-29' (ffff880760c776d8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.394397] kobject: 'rx-29' (ffff880760c776d8): kobject_uevent_env
[  181.394455] kobject: 'rx-29' (ffff880760c776d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-29'
[  181.394568] kobject: 'rx-30' (ffff880760c77758): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.394674] kobject: 'rx-30' (ffff880760c77758): kobject_uevent_env
[  181.394734] kobject: 'rx-30' (ffff880760c77758): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-30'
[  181.394927] kobject: 'rx-31' (ffff880760c777d8): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.395043] kobject: 'rx-31' (ffff880760c777d8): kobject_uevent_env
[  181.395133] kobject: 'rx-31' (ffff880760c777d8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/rx-31'
[  181.395304] kobject: 'tx-0' (ffff880756760018): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.395657] kobject: 'tx-0' (ffff880756760018): kobject_uevent_env
[  181.395727] kobject: 'tx-0' (ffff880756760018): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-0'
[  181.395848] kobject: 'tx-1' (ffff880756760198): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.396170] kobject: 'tx-1' (ffff880756760198): kobject_uevent_env
[  181.396231] kobject: 'tx-1' (ffff880756760198): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-1'
[  181.396349] kobject: 'tx-2' (ffff880756760318): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.396664] kobject: 'tx-2' (ffff880756760318): kobject_uevent_env
[  181.396721] kobject: 'tx-2' (ffff880756760318): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-2'
[  181.396830] kobject: 'tx-3' (ffff880756760498): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.397130] kobject: 'tx-3' (ffff880756760498): kobject_uevent_env
[  181.397186] kobject: 'tx-3' (ffff880756760498): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-3'
[  181.397298] kobject: 'tx-4' (ffff880756760618): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.397617] kobject: 'tx-4' (ffff880756760618): kobject_uevent_env
[  181.397675] kobject: 'tx-4' (ffff880756760618): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-4'
[  181.397786] kobject: 'tx-5' (ffff880756760798): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.398094] kobject: 'tx-5' (ffff880756760798): kobject_uevent_env
[  181.398156] kobject: 'tx-5' (ffff880756760798): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-5'
[  181.398265] kobject: 'tx-6' (ffff880756760918): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.398573] kobject: 'tx-6' (ffff880756760918): kobject_uevent_env
[  181.398646] kobject: 'tx-6' (ffff880756760918): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-6'
[  181.398757] kobject: 'tx-7' (ffff880756760a98): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.399168] kobject: 'tx-7' (ffff880756760a98): kobject_uevent_env
[  181.399231] kobject: 'tx-7' (ffff880756760a98): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-7'
[  181.399349] kobject: 'tx-8' (ffff880756760c18): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.399666] kobject: 'tx-8' (ffff880756760c18): kobject_uevent_env
[  181.399723] kobject: 'tx-8' (ffff880756760c18): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-8'
[  181.399832] kobject: 'tx-9' (ffff880756760d98): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.400132] kobject: 'tx-9' (ffff880756760d98): kobject_uevent_env
[  181.400189] kobject: 'tx-9' (ffff880756760d98): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-9'
[  181.400296] kobject: 'tx-10' (ffff880756760f18): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.400619] kobject: 'tx-10' (ffff880756760f18): kobject_uevent_env
[  181.400678] kobject: 'tx-10' (ffff880756760f18): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-10'
[  181.400791] kobject: 'tx-11' (ffff880756761098): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.401094] kobject: 'tx-11' (ffff880756761098): kobject_uevent_env
[  181.401150] kobject: 'tx-11' (ffff880756761098): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-11'
[  181.401260] kobject: 'tx-12' (ffff880756761218): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.401566] kobject: 'tx-12' (ffff880756761218): kobject_uevent_env
[  181.401623] kobject: 'tx-12' (ffff880756761218): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-12'
[  181.401734] kobject: 'tx-13' (ffff880756761398): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.402050] kobject: 'tx-13' (ffff880756761398): kobject_uevent_env
[  181.402125] kobject: 'tx-13' (ffff880756761398): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-13'
[  181.402238] kobject: 'tx-14' (ffff880756761518): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.402549] kobject: 'tx-14' (ffff880756761518): kobject_uevent_env
[  181.402607] kobject: 'tx-14' (ffff880756761518): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-14'
[  181.402715] kobject: 'tx-15' (ffff880756761698): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.403085] kobject: 'tx-15' (ffff880756761698): kobject_uevent_env
[  181.403177] kobject: 'tx-15' (ffff880756761698): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-15'
[  181.403325] kobject: 'tx-16' (ffff880756761818): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.403654] kobject: 'tx-16' (ffff880756761818): kobject_uevent_env
[  181.403717] kobject: 'tx-16' (ffff880756761818): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-16'
[  181.403828] kobject: 'tx-17' (ffff880756761998): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.404131] kobject: 'tx-17' (ffff880756761998): kobject_uevent_env
[  181.404188] kobject: 'tx-17' (ffff880756761998): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-17'
[  181.404298] kobject: 'tx-18' (ffff880756761b18): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.404620] kobject: 'tx-18' (ffff880756761b18): kobject_uevent_env
[  181.404679] kobject: 'tx-18' (ffff880756761b18): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-18'
[  181.404791] kobject: 'tx-19' (ffff880756761c98): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.405097] kobject: 'tx-19' (ffff880756761c98): kobject_uevent_env
[  181.405153] kobject: 'tx-19' (ffff880756761c98): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-19'
[  181.405264] kobject: 'tx-20' (ffff880756761e18): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.405573] kobject: 'tx-20' (ffff880756761e18): kobject_uevent_env
[  181.405646] kobject: 'tx-20' (ffff880756761e18): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-20'
[  181.405756] kobject: 'tx-21' (ffff880756761f98): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.406072] kobject: 'tx-21' (ffff880756761f98): kobject_uevent_env
[  181.406131] kobject: 'tx-21' (ffff880756761f98): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-21'
[  181.406240] kobject: 'tx-22' (ffff880756762118): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.406554] kobject: 'tx-22' (ffff880756762118): kobject_uevent_env
[  181.406613] kobject: 'tx-22' (ffff880756762118): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-22'
[  181.406727] kobject: 'tx-23' (ffff880756762298): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.407105] kobject: 'tx-23' (ffff880756762298): kobject_uevent_env
[  181.407173] kobject: 'tx-23' (ffff880756762298): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-23'
[  181.407304] kobject: 'tx-24' (ffff880756762418): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.407646] kobject: 'tx-24' (ffff880756762418): kobject_uevent_env
[  181.407705] kobject: 'tx-24' (ffff880756762418): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-24'
[  181.407817] kobject: 'tx-25' (ffff880756762598): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.408124] kobject: 'tx-25' (ffff880756762598): kobject_uevent_env
[  181.408180] kobject: 'tx-25' (ffff880756762598): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-25'
[  181.408288] kobject: 'tx-26' (ffff880756762718): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.408597] kobject: 'tx-26' (ffff880756762718): kobject_uevent_env
[  181.408655] kobject: 'tx-26' (ffff880756762718): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-26'
[  181.408767] kobject: 'tx-27' (ffff880756762898): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.409087] kobject: 'tx-27' (ffff880756762898): kobject_uevent_env
[  181.409157] kobject: 'tx-27' (ffff880756762898): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-27'
[  181.409269] kobject: 'tx-28' (ffff880756762a18): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.409578] kobject: 'tx-28' (ffff880756762a18): kobject_uevent_env
[  181.409636] kobject: 'tx-28' (ffff880756762a18): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-28'
[  181.409745] kobject: 'tx-29' (ffff880756762b98): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.410046] kobject: 'tx-29' (ffff880756762b98): kobject_uevent_env
[  181.410103] kobject: 'tx-29' (ffff880756762b98): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-29'
[  181.410210] kobject: 'tx-30' (ffff880756762d18): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.410530] kobject: 'tx-30' (ffff880756762d18): kobject_uevent_env
[  181.410588] kobject: 'tx-30' (ffff880756762d18): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-30'
[  181.410698] kobject: 'tx-31' (ffff880756762e98): kobject_add_internal: parent: 'queues', set: 'queues'
[  181.411069] kobject: 'tx-31' (ffff880756762e98): kobject_uevent_env
[  181.411145] kobject: 'tx-31' (ffff880756762e98): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0/queues/tx-31'
[  181.426781] nfp 0000:82:00.0 eth0: Netronome NFP-6xxx Netdev: TxQs=20/32 RxQs=8/32
[  181.435750] nfp 0000:82:00.0 eth0: VER: 0.0.4.4, Maximum supported MTU: 9216
[  181.443801] nfp 0000:82:00.0 eth0: CAP: 0x831606ff PROMISC L2BCFILT L2MCFILT RXCSUM TXCSUM RXVLAN TXVLAN GATHER TSO1 RSS1 AUTOMASK IRQMOD VXLAN NVGRE LIVE_ADDR 
[  181.460462] device: 'hwmon0': device_add
[  181.460501] kobject: 'hwmon' (ffff88074f997d48): kobject_add_internal: parent: '0000:82:00.0', set: '(null)'
[  181.460540] kobject: 'hwmon0' (ffff8807616b70a0): kobject_add_internal: parent: 'hwmon', set: 'devices'
[  181.461322] PM: Adding info for No Bus:hwmon0
[  181.461333] kobject: 'hwmon0' (ffff8807616b70a0): kobject_uevent_env
[  181.461400] kobject: 'hwmon0' (ffff8807616b70a0): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/hwmon/hwmon0'
[  181.461681] driver: 'nfp': driver_bound: bound to device '0000:82:00.0'
[  181.461712] kobject: '0000:82:00.0' (ffff880751b480b8): kobject_uevent_env
[  181.461796] kobject: '0000:82:00.0' (ffff880751b480b8): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0'
[  181.461923] bus: 'pci': really_probe: bound device 0000:82:00.0 to driver nfp
[  181.462216] kobject: 'drivers' (ffff8803f03e4688): kobject_add_internal: parent: 'nfp', set: '<NULL>'
[  181.462447] kobject: 'nfp' (ffff880468d650a8): kobject_uevent_env
[  181.462513] kobject: 'nfp' (ffff880468d650a8): fill_kobj_path: path = '/bus/pci/drivers/nfp'
[  181.462610] bus: 'pci': add driver nfp_netvf
[  181.462671] kobject: 'nfp_netvf' (ffff880468d65c88): kobject_add_internal: parent: 'drivers', set: 'drivers'
[  181.463613] kobject: 'nfp_netvf' (ffff880468d65c88): kobject_uevent_env
[  181.463686] kobject: 'nfp_netvf' (ffff880468d65c88): fill_kobj_path: path = '/bus/pci/drivers/nfp_netvf'
[  182.510895] nfp 0000:82:00.0 p4p1: renamed from eth0
[  182.516679] net eth0: renaming to p4p1
[  182.516778] kobject: 'eth0' (ffff8807333d05e0): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/eth0'
[  182.516862] kobject: 'p4p1' (ffff8807333d05e0): kobject_uevent_env
[  182.516919] kobject: 'p4p1' (ffff8807333d05e0): fill_kobj_path: path = '/devices/pci0000:80/0000:80:02.0/0000:82:00.0/net/p4p1'
[  182.548916] systemd-udevd[1607]: renamed network interface eth0 to p4p1
#7 lore raw
Quoted 52 lines · Jakub Kicinski
Quoted 51 lines · Dmitry Torokhov
Quoted 49 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines · Jakub Kicinski
Quoted 3 lines

Hi!

I'm having trouble with modules on linux/master. rmmod succeeds

but the

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1

If I rmmod again the module will be actually unloaded. The user

space

Quoted 1 line
Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying to

bisect

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 7 lines
Quoted 4 lines
Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a driver

Does it happen with all modules or only nfp one?

Quoted 7 lines

It seems to work here:

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded. The driver used to return EPROBE_DEFER, but I think it doesn't any more (rebuilding the kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

Quoted 6 lines
Quoted 1 line

It depends on how silly the udev rules are, but yes, this can definitely happen.

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$

Not sure why you bring the deferrals here, they have nothing to do with module removal. Also, deferrals are rarely issued by the leaf driver, and more often by providers of resources (GPIO, regulator, interrupt, etc).

Quoted 3 lines

I tested without any udev rules in /etc/udev/, just the standard distro ones. Same thing.

Right, so this is the default udev rule:

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}" SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load tifm_sd" SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load tifm_ms" SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block" SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block" SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load ppdev" SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}" SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon" KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load sm_ftl"

LABEL="drivers_end"

So udev (and systemd) want to load kernel module on any action besides device removal. Shortsighted decision I'd say. I'll send a patch to systemd, in the mean time you can simply adjust your local rule to read

ACTION!="add", GOTO="drivers_end"

Thanks.

Signature · 1 line
Dmitry
#8 lore raw
Quoted 55 lines · Dmitry Torokhov
Quoted 54 lines · Jakub Kicinski
Quoted 52 lines · Dmitry Torokhov
Quoted 43 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines · Jakub Kicinski
Quoted 3 lines

Hi!

I'm having trouble with modules on linux/master. rmmod succeeds

but the

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1

If I rmmod again the module will be actually unloaded. The user

space

Quoted 1 line
Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying to

bisect

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 7 lines
Quoted 4 lines
Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a driver

Does it happen with all modules or only nfp one?

Quoted 7 lines

It seems to work here:

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

Quoted 6 lines

It looks like the driver is actually reloaded. The driver used to return EPROBE_DEFER, but I think it doesn't any more (rebuilding the kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

It depends on how silly the udev rules are, but yes, this can definitely happen.

Quoted 8 lines
Quoted 4 lines

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$  

Not sure why you bring the deferrals here, they have nothing to do with module removal. Also, deferrals are rarely issued by the leaf driver, and more often by providers of resources (GPIO, regulator, interrupt, etc).

Yes, it's unusual, but this driver used to do it. Which is exactly why I brought it up. Turns out it was irrelevant :)

Quoted 21 lines
Quoted 2 lines

I tested without any udev rules in /etc/udev/, just the standard distro ones. Same thing.

Right, so this is the default udev rule:

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}" SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load tifm_sd" SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load tifm_ms" SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block" SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block" SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load ppdev" SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}" SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon" KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load sm_ftl"

Quoted 7 lines

LABEL="drivers_end"

So udev (and systemd) want to load kernel module on any action besides device removal. Shortsighted decision I'd say. I'll send a patch to systemd, in the mean time you can simply adjust your local rule to read

ACTION!="add", GOTO="drivers_end"

Mm. That is a silly thing. You will break a lot of setups, though.

#9 lore raw
Quoted 111 lines · Jakub Kicinski
Quoted 3 lines · Dmitry Torokhov
Quoted 2 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski

Quoted 2 lines · <[email protected]> wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Dmitry Torokhov
Quoted 1 line

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines · wrote:
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines

Hi!

I'm having trouble with modules on linux/master. rmmod

succeeds

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

but the

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

If I rmmod again the module will be actually unloaded. The

user

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

space

Quoted 1 line
Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying

to

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

bisect

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 4 lines
Quoted 4 lines
Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a

driver

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Does it happen with all modules or only nfp one?

It seems to work here:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded. The driver used

to

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

return EPROBE_DEFER, but I think it doesn't any more (rebuilding

the

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

It depends on how silly the udev rules are, but yes, this can

definitely happen.

Quoted 7 lines
Quoted 5 lines

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$  

Not sure why you bring the deferrals here, they have nothing to do

with

Quoted 1 line

module removal. Also, deferrals are rarely issued by the leaf driver,

and

Quoted 1 line

more often by providers of resources (GPIO, regulator, interrupt,

etc).

Yes, it's unusual, but this driver used to do it. Which is exactly why I brought it up. Turns out it was irrelevant :)

Quoted 1 line
Quoted 1 line

I tested without any udev rules in /etc/udev/, just the standard

distro

Quoted 4 lines
Quoted 1 line

ones. Same thing.

Right, so this is the default udev rule:

Quoted 8 lines

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}" SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load

tifm_sd"

Quoted 1 line

SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load

tifm_ms"

Quoted 3 lines

SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block" SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block" SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load

ppdev"

Quoted 1 line

SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load

$env{MODALIAS}"

Quoted 2 lines

SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon" KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load

sm_ftl"

Quoted 4 lines

LABEL="drivers_end"

So udev (and systemd) want to load kernel module on any action

Quoted 8 lines

besides

Quoted 2 lines

device removal. Shortsighted decision I'd say. I'll send a patch to systemd, in the mean time you can simply adjust your local rule to

read

Quoted 2 lines

ACTION!="add", GOTO="drivers_end"

Mm. That is a silly thing. You will break a lot of setups, though.

I think the priority it to have module loading working properly, and for most users once module is loaded it stays loaded. Unloading is mostly for developers.

Luckily newer systemd versions drop events they do not recognize, so exposure is even smaller.

Thanks.

Signature · 1 line
Dmitry
#10 lore raw
Quoted 119 lines · Dmitry Torokhov
Quoted 118 lines · Jakub Kicinski
Quoted 3 lines · Dmitry Torokhov
Quoted 2 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski

Quoted 2 lines · <[email protected]> wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Dmitry Torokhov
Quoted 1 line

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines · wrote:
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines

Hi!

I'm having trouble with modules on linux/master. rmmod

succeeds

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

but the

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

If I rmmod again the module will be actually unloaded. The

user

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

space

Quoted 1 line
Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying

to

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

bisect

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 4 lines
Quoted 4 lines
Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a

driver

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Does it happen with all modules or only nfp one?

It seems to work here:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded. The driver used

to

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

return EPROBE_DEFER, but I think it doesn't any more (rebuilding

the

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

It depends on how silly the udev rules are, but yes, this can

definitely happen.

Quoted 7 lines
Quoted 5 lines

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$    

Not sure why you bring the deferrals here, they have nothing to do

with

Quoted 1 line

module removal. Also, deferrals are rarely issued by the leaf driver,

and

Quoted 1 line

more often by providers of resources (GPIO, regulator, interrupt,

etc).

Yes, it's unusual, but this driver used to do it. Which is exactly why I brought it up. Turns out it was irrelevant :)

Quoted 1 line
Quoted 1 line

I tested without any udev rules in /etc/udev/, just the standard

distro

Quoted 4 lines
Quoted 1 line

ones. Same thing.

Right, so this is the default udev rule:

Quoted 8 lines

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}" SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load

tifm_sd"

Quoted 1 line

SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load

tifm_ms"

Quoted 3 lines

SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block" SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block" SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load

ppdev"

Quoted 1 line

SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load

$env{MODALIAS}"

Quoted 2 lines

SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon" KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load

sm_ftl"

Quoted 4 lines

LABEL="drivers_end"

So udev (and systemd) want to load kernel module on any action

besides

Quoted 2 lines

device removal. Shortsighted decision I'd say. I'll send a patch to systemd, in the mean time you can simply adjust your local rule to

read

Quoted 2 lines

ACTION!="add", GOTO="drivers_end"

Quoted 8 lines
Quoted 1 line

Mm. That is a silly thing. You will break a lot of setups, though.

I think the priority it to have module loading working properly, and for most users once module is loaded it stays loaded. Unloading is mostly for developers.

Luckily newer systemd versions drop events they do not recognize, so exposure is even smaller.

Could you point me to where that's done?

#11 lore raw
Quoted 122 lines · Jakub Kicinski
Quoted 121 lines · Dmitry Torokhov
Quoted 119 lines · Jakub Kicinski
Quoted 3 lines · Dmitry Torokhov
Quoted 2 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski

Quoted 2 lines · <[email protected]> wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Dmitry Torokhov
Quoted 1 line

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines · wrote:
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines

Hi!

I'm having trouble with modules on linux/master. rmmod

succeeds

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

but the

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

If I rmmod again the module will be actually unloaded. The

user

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

space

Quoted 1 line
Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying

to

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

bisect

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 4 lines
Quoted 4 lines
Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a

driver

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Does it happen with all modules or only nfp one?

It seems to work here:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded. The driver used

to

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

return EPROBE_DEFER, but I think it doesn't any more (rebuilding

the

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

It depends on how silly the udev rules are, but yes, this can

definitely happen.

Quoted 7 lines
Quoted 5 lines

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$

Not sure why you bring the deferrals here, they have nothing to do

with

Quoted 1 line

module removal. Also, deferrals are rarely issued by the leaf driver,

and

Quoted 1 line

more often by providers of resources (GPIO, regulator, interrupt,

etc).

Yes, it's unusual, but this driver used to do it. Which is exactly why I brought it up. Turns out it was irrelevant :)

Quoted 1 line
Quoted 1 line

I tested without any udev rules in /etc/udev/, just the standard

distro

Quoted 4 lines
Quoted 1 line

ones. Same thing.

Right, so this is the default udev rule:

Quoted 8 lines

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}" SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load

tifm_sd"

Quoted 1 line

SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load

tifm_ms"

Quoted 3 lines

SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block" SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block" SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load

ppdev"

Quoted 1 line

SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load

$env{MODALIAS}"

Quoted 2 lines

SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon" KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load

sm_ftl"

Quoted 4 lines

LABEL="drivers_end"

So udev (and systemd) want to load kernel module on any action

besides

Quoted 2 lines

device removal. Shortsighted decision I'd say. I'll send a patch to systemd, in the mean time you can simply adjust your local rule to

read

Quoted 2 lines

ACTION!="add", GOTO="drivers_end"

Mm. That is a silly thing. You will break a lot of setups, though.

Quoted 8 lines
Quoted 6 lines

I think the priority it to have module loading working properly, and for most users once module is loaded it stays loaded. Unloading is mostly for developers.

Luckily newer systemd versions drop events they do not recognize, so exposure is even smaller.

Could you point me to where that's done?

https://github.com/systemd/systemd/blob/master/src/libsystemd/sd-device/device-private.c#L506
- 508

Signature · 1 line
Dmitry
#12 lore raw
In reply to Dmitry Torokhov
Quoted 88 lines · Dmitry Torokhov
Quoted 87 lines · Jakub Kicinski
Quoted 3 lines · Dmitry Torokhov
Quoted 2 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski

Quoted 2 lines · <[email protected]> wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Dmitry Torokhov
Quoted 1 line

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines · wrote:
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines

Hi!

I'm having trouble with modules on linux/master. rmmod

succeeds

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

but the

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

If I rmmod again the module will be actually unloaded. The

user

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

space

Quoted 1 line
Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying

to

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

bisect

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 4 lines
Quoted 4 lines
Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a

driver

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Does it happen with all modules or only nfp one?

It seems to work here:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded. The driver used

to

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

return EPROBE_DEFER, but I think it doesn't any more (rebuilding

the

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

It depends on how silly the udev rules are, but yes, this can

definitely happen.

Quoted 7 lines
Quoted 5 lines

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$  

Not sure why you bring the deferrals here, they have nothing to do

with

Quoted 1 line

module removal. Also, deferrals are rarely issued by the leaf driver,

and

Quoted 1 line

more often by providers of resources (GPIO, regulator, interrupt,

etc).

Yes, it's unusual, but this driver used to do it. Which is exactly why I brought it up. Turns out it was irrelevant :)

Quoted 1 line
Quoted 1 line

I tested without any udev rules in /etc/udev/, just the standard

distro

Quoted 4 lines
Quoted 1 line

ones. Same thing.

Right, so this is the default udev rule:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"

So if the new uevents do not have the MODALIAS line in them, then they will not trigger this? Dmitry, can you see if that would fix this problem without having to fix everyone's old versions of udev/systemd?

thanks,

greg k-h

#13 lore raw
Quoted 93 lines · Greg Kroah-Hartman
Quoted 92 lines · Dmitry Torokhov
Quoted 91 lines · Jakub Kicinski
Quoted 3 lines · Dmitry Torokhov
Quoted 2 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski

Quoted 2 lines · <[email protected]> wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Dmitry Torokhov
Quoted 1 line

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines · wrote:
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines

Hi!

I'm having trouble with modules on linux/master. rmmod

succeeds

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

but the

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

If I rmmod again the module will be actually unloaded. The

user

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

space

Quoted 1 line
Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying

to

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

bisect

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 4 lines
Quoted 4 lines
Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a

driver

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Does it happen with all modules or only nfp one?

It seems to work here:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded. The driver used

to

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

return EPROBE_DEFER, but I think it doesn't any more (rebuilding

the

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

It depends on how silly the udev rules are, but yes, this can

definitely happen.

Quoted 7 lines
Quoted 5 lines

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$    

Not sure why you bring the deferrals here, they have nothing to do

with

Quoted 1 line

module removal. Also, deferrals are rarely issued by the leaf driver,

and

Quoted 1 line

more often by providers of resources (GPIO, regulator, interrupt,

etc).

Yes, it's unusual, but this driver used to do it. Which is exactly why I brought it up. Turns out it was irrelevant :)

Quoted 1 line
Quoted 1 line

I tested without any udev rules in /etc/udev/, just the standard

distro

Quoted 8 lines
Quoted 1 line

ones. Same thing.

Right, so this is the default udev rule:

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"

So if the new uevents do not have the MODALIAS line in them, then they will not trigger this? Dmitry, can you see if that would fix this problem without having to fix everyone's old versions of udev/systemd?

Perhaps another option is dropping the unbind event? From the commit message it seems like only bind is really needed ATM. Do events have to be symmetrical?

#14 lore raw
Quoted 94 lines · Jakub Kicinski
Quoted 93 lines · Greg Kroah-Hartman
Quoted 92 lines · Dmitry Torokhov
Quoted 91 lines · Jakub Kicinski
Quoted 3 lines · Dmitry Torokhov
Quoted 2 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski

Quoted 2 lines · <[email protected]> wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Dmitry Torokhov
Quoted 1 line

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines · wrote:
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines

Hi!

I'm having trouble with modules on linux/master. rmmod

succeeds

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

but the

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

If I rmmod again the module will be actually unloaded. The

user

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

space

Quoted 1 line
Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying

to

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

bisect

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 4 lines
Quoted 4 lines
Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a

driver

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Does it happen with all modules or only nfp one?

It seems to work here:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded. The driver used

to

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

return EPROBE_DEFER, but I think it doesn't any more (rebuilding

the

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

It depends on how silly the udev rules are, but yes, this can

definitely happen.

Quoted 7 lines
Quoted 5 lines

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$

Not sure why you bring the deferrals here, they have nothing to do

with

Quoted 1 line

module removal. Also, deferrals are rarely issued by the leaf driver,

and

Quoted 1 line

more often by providers of resources (GPIO, regulator, interrupt,

etc).

Yes, it's unusual, but this driver used to do it. Which is exactly why I brought it up. Turns out it was irrelevant :)

Quoted 1 line
Quoted 1 line

I tested without any udev rules in /etc/udev/, just the standard

distro

Quoted 8 lines
Quoted 1 line

ones. Same thing.

Right, so this is the default udev rule:

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"

So if the new uevents do not have the MODALIAS line in them, then they will not trigger this? Dmitry, can you see if that would fix this problem without having to fix everyone's old versions of udev/systemd?

Unfortunately MODALIAS= is being added by individual subsystems having their subsystem specific format. Unless you'd be OK with kobject_uevent_env() poking into the generated environment and zapping MODALIAS= environment variables for KOBJ_BIND/KOBJ_UNBIND actions.

Let me know and I can try to come up with a patch.

I'm still going to submit correction for the rule to systemd folks.

Quoted 4 lines

Perhaps another option is dropping the unbind event? From the commit message it seems like only bind is really needed ATM. Do events have to be symmetrical?

While you are absolutely right that bind is the most important one, I'd be hesitant removing unbind even though we do not have concrete use case for it yet. The bind operation complements unbind, so having bind uevent but not unbind "feels weird".

Thanks.

Signature · 1 line
Dmitry
#15 lore raw
Quoted 99 lines · Dmitry Torokhov
Quoted 98 lines · Jakub Kicinski
Quoted 97 lines · Greg Kroah-Hartman
Quoted 95 lines · Dmitry Torokhov
Quoted 94 lines · Jakub Kicinski
Quoted 3 lines · Dmitry Torokhov
Quoted 2 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski

Quoted 2 lines · <[email protected]> wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Dmitry Torokhov
Quoted 1 line

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines · wrote:
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines

Hi!

I'm having trouble with modules on linux/master. rmmod

succeeds

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

but the

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
      /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
      lsmod | grep nfp; \
      rmmod nfp; \
      lsmod | grep nfp
nfp                   249856  0
nfp                   200704  1
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

If I rmmod again the module will be actually unloaded. The

user

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

space

Quoted 1 line
Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying

to

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

bisect

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 4 lines
Quoted 4 lines
Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a

driver

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Does it happen with all modules or only nfp one?

It seems to work here:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded. The driver used

to

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

return EPROBE_DEFER, but I think it doesn't any more (rebuilding

the

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

It depends on how silly the udev rules are, but yes, this can

definitely happen.

Quoted 7 lines
Quoted 5 lines

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$

Not sure why you bring the deferrals here, they have nothing to do

with

Quoted 1 line

module removal. Also, deferrals are rarely issued by the leaf driver,

and

Quoted 1 line

more often by providers of resources (GPIO, regulator, interrupt,

etc).

Yes, it's unusual, but this driver used to do it. Which is exactly why I brought it up. Turns out it was irrelevant :)

Quoted 1 line
Quoted 1 line

I tested without any udev rules in /etc/udev/, just the standard

distro

Quoted 11 lines
Quoted 1 line

ones. Same thing.

Right, so this is the default udev rule:

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

So if the new uevents do not have the MODALIAS line in them, then they will not trigger this? Dmitry, can you see if that would fix this problem without having to fix everyone's old versions of udev/systemd?

Unfortunately MODALIAS= is being added by individual subsystems having their subsystem specific format. Unless you'd be OK with kobject_uevent_env() poking into the generated environment and zapping MODALIAS= environment variables for KOBJ_BIND/KOBJ_UNBIND actions.

Hm, any reason why it should be sending these values for those uevents? I guess it's not worth hacking around in the lower levels just for this, to work around crazy userspace stuff.

Quoted 1 line

I'm still going to submit correction for the rule to systemd folks.

Yes please.

Quoted 8 lines
Quoted 3 lines

Perhaps another option is dropping the unbind event? From the commit message it seems like only bind is really needed ATM. Do events have to be symmetrical?

While you are absolutely right that bind is the most important one, I'd be hesitant removing unbind even though we do not have concrete use case for it yet. The bind operation complements unbind, so having bind uevent but not unbind "feels weird".

We might want to disable it for a year or so for people to catch up with a newer version of udev/systemd, and then turn it back on?

thanks,

greg k-h

#16 lore raw
Quoted 121 lines · Greg Kroah-Hartman
Quoted 99 lines · Dmitry Torokhov
Quoted 98 lines · Jakub Kicinski
Quoted 97 lines · Greg Kroah-Hartman
Quoted 95 lines · Dmitry Torokhov
Quoted 94 lines · Jakub Kicinski
Quoted 3 lines · Dmitry Torokhov
Quoted 2 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski

Quoted 2 lines · <[email protected]> wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Dmitry Torokhov
Quoted 1 line

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines · wrote:
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines

Hi!

I'm having trouble with modules on linux/master. rmmod

succeeds

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

but the

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
/opt/netronome/bin/nfp-hwinfo -n 2 assembly.partno \
lsmod | grep nfp; \
rmmod nfp; \
lsmod | grep nfp
nfp 249856 0
nfp 200704 1

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

If I rmmod again the module will be actually unloaded. The

user

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

space

Quoted 1 line
Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying

to

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

bisect

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 4 lines
Quoted 4 lines
Author: Dmitry Torokhov <[email protected]>
Date:   Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a

driver

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Does it happen with all modules or only nfp one?

It seems to work here:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded. The driver used

to

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

return EPROBE_DEFER, but I think it doesn't any more (rebuilding

the

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

It depends on how silly the udev rules are, but yes, this can

definitely happen.

Quoted 7 lines
Quoted 5 lines

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$

Not sure why you bring the deferrals here, they have nothing to do

with

Quoted 1 line

module removal. Also, deferrals are rarely issued by the leaf driver,

and

Quoted 1 line

more often by providers of resources (GPIO, regulator, interrupt,

etc).

Yes, it's unusual, but this driver used to do it. Which is exactly why I brought it up. Turns out it was irrelevant :)

Quoted 1 line
Quoted 1 line

I tested without any udev rules in /etc/udev/, just the standard

distro

Quoted 11 lines
Quoted 1 line

ones. Same thing.

Right, so this is the default udev rule:

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

So if the new uevents do not have the MODALIAS line in them, then they will not trigger this? Dmitry, can you see if that would fix this problem without having to fix everyone's old versions of udev/systemd?

Unfortunately MODALIAS= is being added by individual subsystems having their subsystem specific format. Unless you'd be OK with kobject_uevent_env() poking into the generated environment and zapping MODALIAS= environment variables for KOBJ_BIND/KOBJ_UNBIND actions.

Hm, any reason why it should be sending these values for those uevents? I guess it's not worth hacking around in the lower levels just for this, to work around crazy userspace stuff.

Quoted 1 line

I'm still going to submit correction for the rule to systemd folks.

Yes please.

Quoted 4 lines
Quoted 3 lines

Perhaps another option is dropping the unbind event? From the commit message it seems like only bind is really needed ATM. Do events have to be symmetrical?

Quoted 7 lines
Quoted 4 lines

While you are absolutely right that bind is the most important one, I'd be hesitant removing unbind even though we do not have concrete use case for it yet. The bind operation complements unbind, so having bind uevent but not unbind "feels weird".

We might want to disable it for a year or so for people to catch up with a newer version of udev/systemd, and then turn it back on?

That is an option, but maybe we could have the patch below for a year or 2 instead?

Jakub, can you try and see if that works for you?

Signature · 25 lines
Dmitry

driver core: suppress sending MODALIAS in UNBIND uevents

From: Dmitry Torokhov <[email protected]>

The current udev rules cause modules to be loaded on all device events save
for "remove". With the introduction of KOBJ_BIND/KOBJ_UNBIND this causes
issues, as driver modules that have devices bound to their drivers get
immediately reloaded, and it appears to the user that module unloading doe
snot work.

The standard udev matching rule is foillowing:

ENV{MODALIAS}=="?*", RUN{builtin}+="kmod load $env{MODALIAS}"

Given that MODALIAS data is not terribly useful for UNBIND event, let's zap
it from the generated uevent environment until we get userspace updated
with the correct udev rule that only loads modules on "add" event.

Reported-by: Jakub Kicinski <[email protected]>
Signed-off-by: Dmitry Torokhov <[email protected]>
---
 lib/kobject_uevent.c |   47 +++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 39 insertions(+), 8 deletions(-)
lib/kobject_uevent.c +39 -8
index e590523ea476..e5ccec526def 100644
@@ -294,6 +294,24 @@ static void cleanup_uevent_env(struct subprocess_info *info)
 }
 #endif
 
+static void zap_modalias_env(struct kobj_uevent_env *env)
+{
+	int i;
+
+	for (i = 0; i < env->envp_idx;) {
+		if (strcmp(env->envp[i], "MODALIAS=")) {
+			i++;
+			continue;
+		}
+
+		if (i != env->envp_idx - 1)
+			memmove(&env->envp[i], &env->envp[i + 1],
+				sizeof(env->envp[i]) * env->envp_idx - 1);
+
+		env->envp_idx--;
+	}
+}
+
 /**
  * kobject_uevent_env - send an uevent with environmental data
  *
@@ -409,16 +427,29 @@ int kobject_uevent_env(struct kobject *kobj, enum kobject_action action,
 		}
 	}
 
-	/*
-	 * Mark "add" and "remove" events in the object to ensure proper
-	 * events to userspace during automatic cleanup. If the object did
-	 * send an "add" event, "remove" will automatically generated by
-	 * the core, if not already done by the caller.
-	 */
-	if (action == KOBJ_ADD)
+	switch (action) {
+	case KOBJ_ADD:
+		/*
+		 * Mark "add" event so we can make sure we deliver "remove"
+		 * event to userspace during automatic cleanup. If
+		 * the object did send an "add" event, "remove" will
+		 * automatically generated by the core, if not already done
+		 * by the caller.
+		 */
 		kobj->state_add_uevent_sent = 1;
-	else if (action == KOBJ_REMOVE)
+		break;
+
+	case KOBJ_REMOVE:
 		kobj->state_remove_uevent_sent = 1;
+		break;
+
+	case KOBJ_UNBIND:
+		zap_modalias_env(env);
+		break;
+
+	default:
+		break;
+	}
 
 	mutex_lock(&uevent_sock_mutex);
 	/* we will send an event, so request a new sequence number */
#17 lore raw
Quoted 127 lines · Dmitry Torokhov
Quoted 126 lines · Greg Kroah-Hartman
Quoted 99 lines · Dmitry Torokhov
Quoted 98 lines · Jakub Kicinski
Quoted 97 lines · Greg Kroah-Hartman
Quoted 95 lines · Dmitry Torokhov
Quoted 94 lines · Jakub Kicinski
Quoted 3 lines · Dmitry Torokhov
Quoted 2 lines · Jakub Kicinski
Quoted 1 line · Dmitry Torokhov

On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski

Quoted 2 lines · <[email protected]> wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Dmitry Torokhov
Quoted 1 line

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines · wrote:
Quoted 5 lines · Jakub Kicinski

Hi!

I'm having trouble with modules on linux/master. rmmod

succeeds

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

but the

Quoted 10 lines
Quoted 10 lines

module is still loaded and the refcount goes to 1:

#rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
/opt/netronome/bin/nfp-hwinfo -n 2 assembly.partno \
lsmod | grep nfp; \
rmmod nfp; \
lsmod | grep nfp
nfp 249856 0
nfp 200704 1

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

If I rmmod again the module will be actually unloaded. The

user

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

space

Quoted 1 line
Quoted 1 line

is mostly Ubuntu 14.04. Has anyone seen this? I'm trying

to

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

bisect

Quoted 5 lines
Quoted 5 lines

now...

Got 'em!

commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,

refs/bisect/bad)

Quoted 4 lines
Quoted 4 lines

Author: Dmitry Torokhov <[email protected]>
Date: Wed Jul 19 17:24:30 2017 -0700

driver core: emit uevents when device is bound to a

driver

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Does it happen with all modules or only nfp one?

It seems to work here:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse 135168 0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded. The driver used

to

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

return EPROBE_DEFER, but I think it doesn't any more (rebuilding

the

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow else cause the driver to be loaded again?

It depends on how silly the udev rules are, but yes, this can

definitely happen.

Quoted 7 lines
Quoted 5 lines

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$  

Not sure why you bring the deferrals here, they have nothing to do

with

Quoted 1 line

module removal. Also, deferrals are rarely issued by the leaf driver,

and

Quoted 1 line

more often by providers of resources (GPIO, regulator, interrupt,

etc).

Yes, it's unusual, but this driver used to do it. Which is exactly why I brought it up. Turns out it was irrelevant :)

Quoted 1 line
Quoted 1 line

I tested without any udev rules in /etc/udev/, just the standard

distro

Quoted 11 lines
Quoted 1 line

ones. Same thing.

Right, so this is the default udev rule:

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

So if the new uevents do not have the MODALIAS line in them, then they will not trigger this? Dmitry, can you see if that would fix this problem without having to fix everyone's old versions of udev/systemd?

Unfortunately MODALIAS= is being added by individual subsystems having their subsystem specific format. Unless you'd be OK with kobject_uevent_env() poking into the generated environment and zapping MODALIAS= environment variables for KOBJ_BIND/KOBJ_UNBIND actions.

Hm, any reason why it should be sending these values for those uevents? I guess it's not worth hacking around in the lower levels just for this, to work around crazy userspace stuff.

Quoted 1 line

I'm still going to submit correction for the rule to systemd folks.

Yes please.

Quoted 8 lines
Quoted 3 lines

Perhaps another option is dropping the unbind event? From the commit message it seems like only bind is really needed ATM. Do events have to be symmetrical?

While you are absolutely right that bind is the most important one, I'd be hesitant removing unbind even though we do not have concrete use case for it yet. The bind operation complements unbind, so having bind uevent but not unbind "feels weird".

Quoted 7 lines
Quoted 2 lines

We might want to disable it for a year or so for people to catch up with a newer version of udev/systemd, and then turn it back on?

That is an option, but maybe we could have the patch below for a year or 2 instead?

Jakub, can you try and see if that works for you?

Unfortunately this doesn't seem to solve it :(

# modprobe nfp; lsmod | grep nfp; modprobe -r nfp; lsmod | grep nfp
nfp                  1101824  0 
nfp                  1101824  1 
#18 lore raw
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines · Dmitry Torokhov

That is an option, but maybe we could have the patch below for a year or 2 instead?

Quoted 7 lines
Quoted 1 line

Jakub, can you try and see if that works for you?

Unfortunately this doesn't seem to solve it :(

# modprobe nfp; lsmod | grep nfp; modprobe -r nfp; lsmod | grep nfp
nfp                  1101824  0 
nfp                  1101824  1 

Well, I should have tested it before sending out. How about this one?

Signature · 26 lines
Dmitry


driver core: suppress sending MODALIAS in UNBIND uevents

From: Dmitry Torokhov <[email protected]>

The current udev rules cause modules to be loaded on all device events save
for "remove". With the introduction of KOBJ_BIND/KOBJ_UNBIND this causes
issues, as driver modules that have devices bound to their drivers get
immediately reloaded, and it appears to the user that module unloading doe
snot work.

The standard udev matching rule is foillowing:

ENV{MODALIAS}=="?*", RUN{builtin}+="kmod load $env{MODALIAS}"

Given that MODALIAS data is not terribly useful for UNBIND event, let's zap
it from the generated uevent environment until we get userspace updated
with the correct udev rule that only loads modules on "add" event.

Reported-by: Jakub Kicinski <[email protected]>
Signed-off-by: Dmitry Torokhov <[email protected]>
---
 lib/kobject_uevent.c |   49 +++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 41 insertions(+), 8 deletions(-)
lib/kobject_uevent.c +41 -8
index e590523ea476..f237a09a5862 100644
@@ -294,6 +294,26 @@ static void cleanup_uevent_env(struct subprocess_info *info)
 }
 #endif
 
+static void zap_modalias_env(struct kobj_uevent_env *env)
+{
+	static const char modalias_prefix[] = "MODALIAS=";
+	int i;
+
+	for (i = 0; i < env->envp_idx;) {
+		if (strncmp(env->envp[i], modalias_prefix,
+			    sizeof(modalias_prefix) - 1)) {
+			i++;
+			continue;
+		}
+
+		if (i != env->envp_idx - 1)
+			memmove(&env->envp[i], &env->envp[i + 1],
+				sizeof(env->envp[i]) * env->envp_idx - 1);
+
+		env->envp_idx--;
+	}
+}
+
 /**
  * kobject_uevent_env - send an uevent with environmental data
  *
@@ -409,16 +429,29 @@ int kobject_uevent_env(struct kobject *kobj, enum kobject_action action,
 		}
 	}
 
-	/*
-	 * Mark "add" and "remove" events in the object to ensure proper
-	 * events to userspace during automatic cleanup. If the object did
-	 * send an "add" event, "remove" will automatically generated by
-	 * the core, if not already done by the caller.
-	 */
-	if (action == KOBJ_ADD)
+	switch (action) {
+	case KOBJ_ADD:
+		/*
+		 * Mark "add" event so we can make sure we deliver "remove"
+		 * event to userspace during automatic cleanup. If
+		 * the object did send an "add" event, "remove" will
+		 * automatically generated by the core, if not already done
+		 * by the caller.
+		 */
 		kobj->state_add_uevent_sent = 1;
-	else if (action == KOBJ_REMOVE)
+		break;
+
+	case KOBJ_REMOVE:
 		kobj->state_remove_uevent_sent = 1;
+		break;
+
+	case KOBJ_UNBIND:
+		zap_modalias_env(env);
+		break;
+
+	default:
+		break;
+	}
 
 	mutex_lock(&uevent_sock_mutex);
 	/* we will send an event, so request a new sequence number */
author Tested-by
#19 lore raw
Quoted 8 lines · Dmitry Torokhov
Quoted 7 lines · Jakub Kicinski
Quoted 5 lines · Dmitry Torokhov

That is an option, but maybe we could have the patch below for a year or 2 instead?

Jakub, can you try and see if that works for you?

Quoted 7 lines
Quoted 5 lines

Unfortunately this doesn't seem to solve it :(

# modprobe nfp; lsmod | grep nfp; modprobe -r nfp; lsmod | grep nfp
nfp                  1101824  0 
nfp                  1101824  1   

Well, I should have tested it before sending out. How about this one?

This one is better! Consider it:

Thanks!