Reduce noise about missing devices.
RFC PATCH 1/1 Reduce noise about missing devices.
Hi,
this is not intended to be directly applied, but more to start a discussion. One problem with our handling of missing devices is the huge amount of log noise each missing device produces. In a single run of lvconvert --repair, I get multiple screens of "read error" and "Couldn't find device with uuid"...
The attached patch reduces that noise by only reporting these things once per run. I have used two different approaches for read errors and for uuid's. Neither is perfect...
Another possible approach (one that I would probably prefer) is to add a global log_error_once to log.c that would maintain a dm_hash_table of error messages and downgrade already-reported errors to log_debug or log_very_verbose automatically. A few error paths could use this.
The downside of all these approaches is that this is sort of global knowledge, but I guess that's hard to avoid. Presumably, adding a call to erase the hashtable to _init_logging would solve most of the problems with this. I suspect that threading a toolcontext pointer through everything so it could be passed to logging functions is not a viable option.
About the device read errors, I am ambivalent what exactly to report as log_error. It makes sense to only report first error per device, but it also makes sense to report all unique errors per device. The global log_error_once approach would give the latter, which I think may be actually preferable.
Opinions?
Yours,
Petr.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: reduce-dup-errors.diff
Type: text/x-diff
Size: 3559 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/lvm-devel/attachments/20100427/824c0fca/attachment.bin>
RFC PATCH 1/1 Reduce noise about missing devices.
Hi,
Quoted 4 lines · Petr Rockai
Another possible approach (one that I would probably prefer) is to add a global
log_error_oncetolog.cthat would maintain adm_hash_tableof error messages and downgrade already-reported errors tolog_debugorlog_very_verboseautomatically. A few error paths could use this.
here's a patch that implements this. I think this is fairly reasonable and could be checked in. Some minor function renaming might be necessary.
Yours,
Petr.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: log_error_once.diff
Type: text/x-diff
Size: 5980 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/lvm-devel/attachments/20100428/a4a1610f/attachment.bin>
RFC PATCH 1/1 Reduce noise about missing devices.
In reply to Petr RockaiQuoted 5 lines · Peter Rockai
this is not intended to be directly applied, but more to start a discussion. One problem with our handling of missing devices is the huge amount of log noise each missing device produces. In a single run of lvconvert --repair, I get multiple screens of "read error" and "Couldn't find device with uuid"...
Well my first question before suppressing messages would be: Are all the repeated accesses that generate the messages actually necessary? Or are the messages just a symptom of other problems which should be addressed instead?
Alasdair
RFC PATCH 1/1 Reduce noise about missing devices.
Quoted 11 lines · Alasdair G Kergon
Quoted 5 lines · Peter Rockai
this is not intended to be directly applied, but more to start a discussion. One problem with our handling of missing devices is the huge amount of log noise each missing device produces. In a single run of lvconvert --repair, I get multiple screens of "read error" and "Couldn't find device with uuid"...
Well my first question before suppressing messages would be: Are all the repeated accesses that generate the messages actually necessary? Or are the messages just a symptom of other problems which should be addressed instead?
I checked how many times a broken device was accessed by the lvconvert command when 'lvconvert --repair' is executed using the following lvm environment:
VG: 3PVs
LV: mirror composed of 2 legs and 1 disk log
vg00-lv00 (253:3)
|-vg00-lv00_mimage_1 (253:2)
| `- (8:48)
|-vg00-lv00_mimage_0 (253:1)
| `- (8:32)
`-vg00-lv00_mlog (253:0)
`- (8:64)When a primary leg is broken, 'lvconvert --repair' accesses the primary leg by '47 times.' I think what we should do is to suppress 'access to a broken device' instead of suppressing 'displaying error messages.'
Suppress error messages might be friendly to some users, but other users might not notice how often a broken device is accessed by the lvconvert command.
Do we need to access a broken device so many times? If lvm commands detects errors on a device a few times, my opinion is that lvm commands should quit accessing the device.
Thanks,
Taka
RFC PATCH 1/1 Reduce noise about missing devices.
Quoted 3 lines · Takahiro Yasui
Do we need to access a broken device so many times? If lvm commands detects errors on a device a few times, my opinion is that lvm commands should quit accessing the device.
Agree.
Alasdair