Implement event loss prevention for db output
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest4 patches 0 repliesNew
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/4 | postgresql: add sanity checkingoutput/ | +3 | 0 replies | |
| 2/4 | mysql: add sanity checkingoutput/ | +3 | 0 replies | |
| 3/4 | sqlite3: add sanity checkingoutput/ | +2 | 0 replies | |
| 4/4 | db: store data in memory during database downtimeinclude/ ulogd.conf.in util/ | +180 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [Ulogd PATCH 0/4] Implement event loss prevention for db output
Hello,
The first three patches are cleaning that could be necessary to avoid issue with the fourth one who does the real job.
The idea is to store the queries in memory if the database goes down and to proceed to the insertion when it gets back up. To do so, a chained list of queries is attached to the database instance and queries are added to it till a memory cap is not reached. When the database came back, the queries are played in order so it should not cause any issue for any potential triggers.
This new feature is inactive by default and can be activated by setting "backlog_memcap" to a non null value in the database instance configuration.
To avoid to cause overun on the event capture by inserting too many things at once at recovery (and thus blocking capture for a certain time), a maximum of "backlog_oneshot_requests" " are inserted at a time.
Patchset statistics:
include/ulogd/db.h | 34 ++++++++++-- output/mysql/ulogd_output_MYSQL.c | 4 +- output/pgsql/ulogd_output_PGSQL.c | 4 +- output/sqlite3/ulogd_output_SQLITE3.c | 2 + ulogd.conf.in | 9 +++ util/db.c | 170 +++++++++++++++++++++++++++++++++++++++++++++++---------- 6 files changed, 188 insertions(+), 35 deletions(-)
BR,
--
Eric Leblond <[email protected]>