Skip to contentSkip to conversation

ndctl: nvdimmd: notify/monitor the feathers of over threshold event

RFC PATCH Superseded QI Fuli 3 participants 20 messages Nov 14, 2017 – Dec 9, 2017

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

QI Fuli

author
#0 lore raw

v2 cover ndctl: nvdimmd: notify/monitor the feathers of over threshold event

Hi, here is my second version of nvdimm daemon, It would be appreciated if you could check it.

Change log since v1:
- Adding a config file(/etc/nvdimmd/nvdimmd.conf)
- Using struct log_ctx instead of syslog()
- Using log_syslog() to save the notify messages to syslog
- Using log_file() to save the notify messages to special file
- Adding LOG_NOTICE level into log_priority
- Using automake instead of Makefile
- Adding a new util file(nvdimmd/util.c) including helper functions needed for nvdimm daemon.
- Adding nvdimmd_test program

---
This is a patch set of nvdimmd, a tiny daemon to monitor the features of over threshold events. When an over thershold event fires, nvdimmd will output the notification including dimm health status to syslog or a special file to users' configuration. Users can choose the output format to be structured json or text.

Here are out put samples.
- json format:
2017/11/10 11:15:03 [28065] log_notify: nvdimm dimm over threshold notify {
"dev":"nmem1",
"id":"cdab-0a-07e0-feffffff",
"handle":1,
"phys_id":1,

  "health":{
    "health_state":"non-critical",
    "temperature_celsius":23,
    "spares_percentage":75,
    "alarm_temperature":true,
    "alarm_spares":true,
    "temperature_threshold":40,
    "spares_threshold":5,
    "life_used_percentage":5,
    "shutdown_state":"clean"

}
}
- text format:
2017/11/10 16:21:53 [12479] log_notify: nvdimm dimm over threshold notify
dev: nmem1
health_state: non-critical
spares_percentage: 75

TODO list:
- The dimms to monitor should be filtered by namespace and region
- Add more information into the notify message
- Make nvdimmd_test an ndctl command or an option of ndctl inject-error

QI Fuli (7):
ndctl: nvdimmd: add LOG_NOTICE level into log_priority
ndctl: nvdimmd: add nvdimmd necessary util functions
ndctl: nvdimmd: add nvdimmd necessary functions
ndctl: nvdimmd: add body file of nvdimm daemon
ndctl: nvdimmd: add nvdimmd config file
ndctl: nvdimmd: add the unit file of systemd for nvdimmd service
ndctl: nvdimmd: add a temporary test for nvdimm daemon

 Makefile.am             |   2 +-
 configure.ac            |   1 +
 nvdimmd/Makefile.am     |  47 ++++++++
 nvdimmd/libnvdimmd.c    | 315 ++++++++++++++++++++++++++++++++++++++++++++++++
 nvdimmd/libnvdimmd.h    |  53 ++++++++
 nvdimmd/nvdimmd.c       | 112 +++++++++++++++++
 nvdimmd/nvdimmd.conf    |  25 ++++
 nvdimmd/nvdimmd.service |   7 ++
 nvdimmd/nvdimmd_test.c  | 142 ++++++++++++++++++++++
 nvdimmd/util.c          |  80 ++++++++++++
 nvdimmd/util.h          |  33 +++++
 util/log.c              |   2 +
 util/log.h              |   3 +
 13 files changed, 821 insertions(+), 1 deletion(-)
 create mode 100644 nvdimmd/Makefile.am
 create mode 100644 nvdimmd/libnvdimmd.c
 create mode 100644 nvdimmd/libnvdimmd.h
 create mode 100644 nvdimmd/nvdimmd.c
 create mode 100644 nvdimmd/nvdimmd.conf
 create mode 100644 nvdimmd/nvdimmd.service
 create mode 100644 nvdimmd/nvdimmd_test.c
 create mode 100644 nvdimmd/util.c
 create mode 100644 nvdimmd/util.h
Signature · 7 lines
2.9.5


_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
author
#1 lore raw

RFC PATCH v2 1/7 ndctl: nvdimmd: add LOG_NOTICE level into log_priority

This patch adds LOG_NOTICE level into log_priority for the notifications. Because the LOG_INFO level is too low for the notifications of nvdimm dimm over threshold event, and the event notifications are not up to LOG_ERROR level.

---
 util/log.c | 2 ++
 util/log.h | 3 +++
 2 files changed, 5 insertions(+)
util/log.c +2 -0
index cc4d7d8..c60ca33 100644
@@ -50,6 +50,8 @@ static int log_priority(const char *priority)
 		return LOG_INFO;
 	if (strncmp(priority, "debug", 5) == 0)
 		return LOG_DEBUG;
+	if (strncmp(priority, "notice", 6) == 0)
+		return LOG_NOTICE;
 	return 0;
 }
 
util/log.h +3 -0
index 5fc56e8..495e0d3 100644
@@ -48,15 +48,18 @@ do { \
 #  endif
 #  define log_info(ctx, arg...) log_cond(ctx, LOG_INFO, ## arg)
 #  define log_err(ctx, arg...) log_cond(ctx, LOG_ERR, ## arg)
+#  define log_notice(ctx, arg...) log_cond(ctx, LOG_NOTICE, ## arg)
 #else
 #  define log_dbg(ctx, arg...) log_null(ctx, ## arg)
 #  define log_info(ctx, arg...) log_null(ctx, ## arg)
 #  define log_err(ctx, arg...) log_null(ctx, ## arg)
+#  define log_notice(ctx, arg...) log_null(ctx, ## arg)
 #endif
 
 #define dbg(x, arg...) log_dbg(&(x)->ctx, ## arg)
 #define info(x, arg...) log_info(&(x)->ctx, ## arg)
 #define err(x, arg...) log_err(&(x)->ctx, ## arg)
+#define notice(x, arg...) log_notice(&(x)->ctx, ## arg)
 
 #ifndef HAVE_SECURE_GETENV
 #  ifdef HAVE___SECURE_GETENV
Signature · 7 lines
2.9.5


_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
author
#2 lore raw

RFC PATCH v2 2/7 ndctl: nvdimmd: add nvdimmd necessary util functions

In reply to QI Fuli

This patch is used to provide helper functions needed for nvdimm daemon. These util functions can be used by other features as well in the future.

---
 nvdimmd/util.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 nvdimmd/util.h | 33 ++++++++++++++++++++++++
 2 files changed, 113 insertions(+)
 create mode 100644 nvdimmd/util.c
 create mode 100644 nvdimmd/util.h
nvdimmd/util.c new file +80 -0
new file mode 100644
index 0000000..ef6819e
@@ -0,0 +1,80 @@
+/*
+ * Copyright (c) 2017, FUJITSU LIMITED. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU Lesser General Public License,
+ * version 2.1, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT ANY
+ * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for
+ * more details.
+ */
+
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include "util.h"
+
+void logrotate(char *file)
+{
+	time_t c_time;
+	char buf[4096], tstamp[32];
+
+	c_time = time(NULL);
+	strftime(tstamp, sizeof(tstamp), ".%Y%m%d%H%M%S", localtime(&c_time));
+	strcpy(buf, file);
+	strcat(buf, tstamp);
+	rename(file, buf);
+}
+
+long get_size(char *file)
+{
+	struct stat buf;
+	if (stat(file, &buf) == 0)
+		return buf.st_size;
+	return -1L;
+}
+
+char *set_string(char **str, const char *value)
+{
+	*str = strdup(value);
+	return *str;
+}
+
+char *trim_string(char *str)
+{
+	char *start;
+	char *end;
+	int len = strlen(str);
+
+	if (len == 0)
+		return NULL;
+
+	if (str[len-1] == '\n') {
+		len--;
+		str[len] = 0;
+	}
+
+	start = str;
+	end = str + len - 1;
+	while(*start && isspace(*start))
+		start++;
+	while(*end && isspace(*end))
+		*end-- = 0;
+	strcpy(str, start);
+	return str;
+}
+
+int split_string(char *str, const char *deli, char *outlist[])
+{
+	int cnt = 0;
+	char *temp;
+
+	temp = strtok(str, deli);
+	while (temp != NULL && cnt < NUM_MAX_DIMM) {
+		outlist[cnt++] = temp;
+		temp = strtok(NULL, deli);
+	}
+	return cnt;
+}
nvdimmd/util.h new file +33 -0
new file mode 100644
index 0000000..9648256
@@ -0,0 +1,33 @@
+/*
+ * Copyright (c) 2017, FUJITSU LIMITED. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU Lesser General Public License,
+ * version 2.1, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT ANY
+ * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for
+ * more details.
+ */
+
+#ifndef _UTIL_H_
+#define _UTIL_H_
+
+#include <stdio.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <ctype.h>
+#define NUM_MAX_DIMM 1024
+
+void logrotate(char *file);
+
+long get_size(char *file);
+
+char *set_string(char **str, const char *value);
+
+char *trim_string(char *str);
+
+int split_string(char *str, const char *deli, char *outlist[]);
+
+#endif
Signature · 7 lines
2.9.5


_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
#3 lore raw

Re: [RFC PATCH v2 2/7] ndctl: nvdimmd: add nvdimmd necessary util functions

Quoted 36 lines · QI Fuli

This patch is used to provide helper functions needed for nvdimm daemon. These util functions can be used by other features as well in the future.

---
 nvdimmd/util.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 nvdimmd/util.h | 33 ++++++++++++++++++++++++
 2 files changed, 113 insertions(+)
 create mode 100644 nvdimmd/util.c
 create mode 100644 nvdimmd/util.h
nvdimmd/util.c new file +19 -0
new file mode 100644
index 0000000..ef6819e
@@ -0,0 +1,80 @@
+/*
+ * Copyright (c) 2017, FUJITSU LIMITED. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU Lesser General Public License,
+ * version 2.1, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT ANY
+ * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for
+ * more details.
+ */
+
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include "util.h"
+
+void logrotate(char *file)

nvdimmd/util.c line 29

Quoted 10 lines
nvdimmd/util.c new file +10 -0
+{
+       time_t c_time;
+       char buf[4096], tstamp[32];
+
+       c_time = time(NULL);
+       strftime(tstamp, sizeof(tstamp), ".%Y%m%d%H%M%S", localtime(&c_time));
+       strcpy(buf, file);
+       strcat(buf, tstamp);
+       rename(file, buf);
+}

Why does the monitor need to rotate logs? This should be the responsibility of whatever is consuming the nvdimmd events.
_______________________________________________
Linux-nvdimm mailing list [email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

author
#4 lore raw

Re: [RFC PATCH v2 2/7] ndctl: nvdimmd: add nvdimmd necessary util functions

Quoted 41 lines · Dan Williams
Quoted 40 lines · QI Fuli

This patch is used to provide helper functions needed for nvdimm daemon. These util functions can be used by other features as well in the future.

---
  nvdimmd/util.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  nvdimmd/util.h | 33 ++++++++++++++++++++++++
  2 files changed, 113 insertions(+)

create mode 100644 nvdimmd/util.c
create mode 100644 nvdimmd/util.h

nvdimmd/util.c new file +23 -0
new file mode 100644
index 0000000..ef6819e
@@ -0,0 +1,80 @@
+/*
+ * Copyright (c) 2017, FUJITSU LIMITED. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU Lesser General Public License,
+ * version 2.1, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT ANY
+ * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for
+ * more details.
+ */
+
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include "util.h"
+
+void logrotate(char *file)
+{
+       time_t c_time;
+       char buf[4096], tstamp[32];
+
Quoted 8 lines

nvdimmd/util.c line 29

Quoted 6 lines
nvdimmd/util.c new file +6 -0
+       c_time = time(NULL);
+       strftime(tstamp, sizeof(tstamp), ".%Y%m%d%H%M%S", localtime(&c_time));
+       strcpy(buf, file);
+       strcat(buf, tstamp);
+       rename(file, buf);
+}

Why does the monitor need to rotate logs? This should be the responsibility of whatever is consuming the nvdimmd events.

When I wrote it, I thought that users can rotate nvdimmd logs without using other tools. If you think it is not necessary, I will remove it in next version.

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#5 lore raw

Re: [RFC PATCH v2 2/7] ndctl: nvdimmd: add nvdimmd necessary util functions

Quoted 61 lines · Fuli
Quoted 51 lines · Dan Williams
Quoted 49 lines · QI Fuli

This patch is used to provide helper functions needed for nvdimm daemon. These util functions can be used by other features as well in the future.

---
  nvdimmd/util.c | 80
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  nvdimmd/util.h | 33 ++++++++++++++++++++++++
  2 files changed, 113 insertions(+)

create mode 100644 nvdimmd/util.c
create mode 100644 nvdimmd/util.h

nvdimmd/util.c new file +4 -0
new file mode 100644
index 0000000..ef6819e
@@ -0,0 +1,80 @@
+/*
+ * Copyright (c) 2017, FUJITSU LIMITED. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify

it
+ * under the terms and conditions of the GNU Lesser General Public License,
+ * version 2.1, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT ANY
+ * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for
+ * more details.
+ */

nvdimmd/util.c new file +13 -0
+
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include "util.h"
+
+void logrotate(char *file)
+{
+       time_t c_time;
+       char buf[4096], tstamp[32];
+
+       c_time = time(NULL);
+       strftime(tstamp, sizeof(tstamp), ".%Y%m%d%H%M%S",
Quoted 8 lines

nvdimmd/util.c line 29

Quoted 5 lines

localtime(&c_time));

nvdimmd/util.c new file +4 -0
+       strcpy(buf, file);
+       strcat(buf, tstamp);
+       rename(file, buf);
+}

Why does the monitor need to rotate logs? This should be the responsibility of whatever is consuming the nvdimmd events.

Quoted 4 lines

When I wrote it, I thought that users can rotate nvdimmd logs without using other tools.
If you think it is not necessary, I will remove it in next version.

In general we should let other system components handle the mechanics they are responsible to handle. In this case as long as nvdimmd reports the log message to the systemd-journal or the platform's syslog facility then it has done its job and the log machinery is responsible for rotation.

I've also been giving more thought about the command line interface for the monitor. I think it should reuse the "ndctl list" infrastructure for selecting and filtering devices so the user can launch monitors like this:

        ndctl monitor --dimms --namespace=namespace0.0 --action=page_admin.sh
        ndctl monitor -BDRN --action=log

Where those commands would run the 'page admin' script if any DIMM event happens on any DIMM associated with namespace0.0, and log everything else" that happens on any other device. We likely also need "--event=" to control the events to filter, and a --list-events option to enumerate all the possible events.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

author
#6 lore raw

Re: [RFC PATCH v2 2/7] ndctl: nvdimmd: add nvdimmd necessary util functions

Quoted 20 lines
Quoted 9 lines
Quoted 5 lines

Review

Quoted 3 lines
(quoted diff) +2 -0
+       rename(file, buf);
+}

Why does the monitor need to rotate logs? This should be the responsibility of whatever is consuming the nvdimmd events.

When I wrote it, I thought that users can rotate nvdimmd logs without using other tools.
If you think it is not necessary, I will remove it in next version.

In general we should let other system components handle the mechanics they are responsible to handle. In this case as long as nvdimmd reports the log message to the systemd-journal or the platform's syslog facility then it has done its job and the log machinery is responsible for rotation.

I've also been giving more thought about the command line interface for the monitor. I think it should reuse the "ndctl list" infrastructure for selecting and filtering devices so the user can launch monitors like this:

Quoted 10 lines
         ndctl monitor --dimms --namespace=namespace0.0 --action=page_admin.sh
         ndctl monitor -BDRN --action=log

Where those commands would run the 'page admin' script if any DIMM event happens on any DIMM associated with namespace0.0, and log everything else" that happens on any other device. We likely also need "--event=" to control the events to filter, and a --list-events option to enumerate all the possible events.

My original plan was user launches the nvdimm monitor services with systemctl command
like: $systemctl start nvdimmd, then nvdimmd reads a config file
to select and filter devices or any other settings.

In my first impression, your idea, which is by command line options, seems to be attractive. But I'm not sure yet.

So, I have some questions.

Q1) Which is better interface config file or command option?

    For example, please consider a use-case which user execute plural nvdimm daemons to monitor plural areas.

   - config file
     To make plural daemons, user need to prepare different name of config files.
     But, user can manage each daemon with the name of files.

   - ndctl
     It is easy to make environment plural nvdimm daemons. However, I could not find the way for users to manage them. How to specify each daemon to stop/restart them?

Q2) What filter option is necessary?
   a) filter by dimm
   b) filter by bus
   c) filter by region
   d) filter by namespace
   e) filter by device name
   f) filter by event
   g) all (default)

  a) is already implemented because it is easy.
     (However, to be honest, I don't have any concrete use-case. Do you have any idea?)
  b) is same with a). I don't know any use-case about bus-filter. (any use-case?)
  c), d) or e) will be probably necessary.
    Probably, I think it is useful to notify each applications which uses each regions/namespace.
    However, I feel I need more time to investigate how to implement it yet.
  f) I don't have idea what kind of event should be filtered. Currently, I can think the followings
        - spare threshold event
        - temperature threshold event
    Would you mind elaborating a bit more?
  g) is essential of cause.

BTW, Currently, I would like to merge a), b), e), f) at this time as a first version of NVDIMM daemon. and make c) and d) at next version.
About actions, I think more research and discussion is needed,
so I will only implement output to log in first version.

How do you think?
Thanks

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#7 lore raw

Re: [RFC PATCH v2 2/7] ndctl: nvdimmd: add nvdimmd necessary util functions

Quoted 54 lines · Fuli
Quoted 27 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
(quoted diff) +2 -0
+       rename(file, buf);
+}
Quoted 8 lines
Quoted 2 lines

Why does the monitor need to rotate logs? This should be the responsibility of whatever is consuming the nvdimmd events.

When I wrote it, I thought that users can rotate nvdimmd logs without using
other tools.
If you think it is not necessary, I will remove it in next version.

In general we should let other system components handle the mechanics they are responsible to handle. In this case as long as nvdimmd reports the log message to the systemd-journal or the platform's syslog facility then it has done its job and the log machinery is responsible for rotation.

I've also been giving more thought about the command line interface for the monitor. I think it should reuse the "ndctl list" infrastructure for selecting and filtering devices so the user can launch monitors like this:

         ndctl monitor --dimms --namespace=namespace0.0
--action=page_admin.sh
         ndctl monitor -BDRN --action=log
Quoted 7 lines

Where those commands would run the 'page admin' script if any DIMM event happens on any DIMM associated with namespace0.0, and log everything else" that happens on any other device. We likely also need "--event=" to control the events to filter, and a --list-events option to enumerate all the possible events.

My original plan was user launches the nvdimm monitor services with systemctl command
like: $systemctl start nvdimmd, then nvdimmd reads a config file to select and filter devices or any other settings.

In my first impression, your idea, which is by command line options, seems to be attractive. But I'm not sure yet.

So, I have some questions.

Q1) Which is better interface config file or command option?

For example, please consider a use-case which user execute plural nvdimm daemons to monitor plural areas.

- config file
To make plural daemons, user need to prepare different name of config files.
But, user can manage each daemon with the name of files.

Quoted 4 lines

- ndctl
It is easy to make environment plural nvdimm daemons. However, I could not find the way for users to manage them. How to specify each daemon to stop/restart them?

Every option that can be specified via a configuration file should also be enabled via a command line operation, see dnsmasq as an example.

The systemd service file can launch a single instance to use a default configuration file, but that's just a default that the user is free to ignore and they can always launch monitor instances by hand for testing or custom monitoring purposes.

Quoted 13 lines

Q2) What filter option is necessary?
a) filter by dimm
b) filter by bus
c) filter by region
d) filter by namespace
e) filter by device name
f) filter by event
g) all (default)

a) is already implemented because it is easy. (However, to be honest, I don't have any concrete use-case. Do you have any idea?)
b) is same with a). I don't know any use-case about bus-filter.

Quoted 5 lines
     (any use-case?)

c), d) or e) will be probably necessary.
Probably, I think it is useful to notify each applications which uses each regions/namespace.
However, I feel I need more time to investigate how to implement it yet.

You don't need to implement it. All of this filtering is already present in the 'ndctl list' command. We just need to refactor the code that walks and filters the device tree to be re-used by the monitor implementation.

Quoted 5 lines

f) I don't have idea what kind of event should be filtered. Currently, I can think the followings
- spare threshold event
- temperature threshold event
Would you mind elaborating a bit more?

Every event that we can monitor needs a name and a corresponding event record format in json. The event list I can think of is:

dimm-spares-remaining
dimm-media-temperature
dimm-controller-temperature
dimm-health-state
dimm-unclean-shutdown
dimm-detected
namespace-media-error
namespace-detected
region-media-error
region-detected
bus-media-error
bus-address-range-scrub-complete
bus-detected

Any others?
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

author
#8 lore raw

RFC PATCH v2 3/7 ndctl: nvdimmd: add nvdimmd necessary functions

In reply to QI Fuli

Libnvdimmd is used to provide nvdimmd necessary functions. The get_nvdimmd_conf() will read the nvdimmd.conf and set the parameters' value as global variables when nvdimmd service starts.

---
 nvdimmd/libnvdimmd.c | 315 +++++++++++++++++++++++++++++++++++++++++++++++++++
 nvdimmd/libnvdimmd.h |  53 +++++++++
 2 files changed, 368 insertions(+)
 create mode 100644 nvdimmd/libnvdimmd.c
 create mode 100644 nvdimmd/libnvdimmd.h
nvdimmd/libnvdimmd.c new file +315 -0
new file mode 100644
index 0000000..7e5d16b
@@ -0,0 +1,315 @@
+/*
+ * Copyright (c) 2017, FUJITSU LIMITED. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU Lesser General Public License,
+ * version 2.1, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT ANY
+ * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for
+ * more details.
+ */
+
+/*
+ * This program is used to provide nvdimm daemon necessary functions.
+ */
+
+#include <syslog.h>
+#include <json-c/json.h>
+#include <util/json.h>
+#include <util/filter.h>
+#include <ndctl/lib/private.h>
+#include <dirent.h>
+#include "libnvdimmd.h"
+
+void log_syslog(struct ndctl_ctx *ctx, int priority, const char *file,
+	int line, const char *fn, const char *format, va_list args)
+{
+	char buf[BUF_SIZE];
+	struct log_ctx *log_ctx;
+
+	log_ctx = &ctx->ctx;
+	vsnprintf(buf, sizeof(buf), format, args);
+	syslog(priority, "%s: %s: %s", log_ctx->owner, fn, buf);
+}
+
+void log_file(struct ndctl_ctx *ctx, int priority, const char *file,
+	int line, const char *fn, const char *format, va_list args)
+{
+	DIR *dir;
+	FILE *f;
+	time_t c_time;
+	char date[32], buf[BUF_SIZE];
+	char *filename;
+	int len = strlen(nvdimmd_cf->logfile) + strlen("/var/log/nvdimmd/");
+
+	filename = (char *) malloc(len);
+	strcpy(filename, "/var/log/nvdimmd/");
+	strcat(filename, nvdimmd_cf->logfile);
+
+	dir = opendir("/var/log/nvdimmd");
+	if (dir) {
+		if (get_size(filename) >= 20000000)
+			logrotate(filename);
+	} else {
+		if (mkdir("/var/log/nvdimmd", 0744) == -1) {
+			syslog(LOG_ERR, "cannot make dir /var/log/nvdimmd\n");
+			exit(EXIT_FAILURE);
+		}
+	}
+
+	f = fopen(filename, "a+");
+	if (f == NULL) {
+		syslog(LOG_ERR, "%s cannot be opened\n", filename);
+		exit(EXIT_FAILURE);
+	}
+
+	c_time = time(NULL);
+	strftime(date, sizeof(date), "%Y/%m/%d %H:%M:%S", localtime(&c_time));
+	vsnprintf(buf, sizeof(buf), format, args);
+	fprintf(f, "%s [%u] %s: %s\n", date, (int)getpid(), fn, buf);
+	fclose(f);
+	free(filename);
+}
+
+static char *get_json_msg(struct ndctl_ctx *ctx, struct threshold_dimm *t_dimm)
+{
+	struct json_object *jdimm, *jhealth;
+	char *msg;
+
+	msg = (char *) malloc(1024);
+	msg[0] = '\0';
+	jdimm = util_dimm_to_json(t_dimm->dimm, 0);
+	if (!jdimm) {
+		printf("no jdimm");
+	}
+	jhealth = util_dimm_health_to_json(t_dimm->dimm);
+	if (jhealth)
+		json_object_object_add(jdimm, "health", jhealth);
+	else if (ndctl_dimm_is_cmd_supported(t_dimm->dimm, ND_CMD_SMART))
+		dbg(ctx, "dimm [%s] do not support smart", t_dimm->devname);
+	else
+		dbg(ctx, "dimm [%s] can not get health info", t_dimm->devname);
+
+	strcat(msg, json_object_to_json_string_ext(jdimm,
+			JSON_C_TO_STRING_PRETTY));
+	return msg;
+}
+
+static char *get_text_msg(struct ndctl_ctx *ctx, struct threshold_dimm *t_dimm)
+{
+	char *msg;
+	struct ndctl_cmd *cmd;
+	int rc;
+	unsigned int flags;
+
+	msg = (char *) malloc(1024);
+	msg[0] = '\0';
+	strcat(msg, "dev: ");
+	strcat(msg, t_dimm->devname);
+	strcat(msg, "\n health_state: ");
+
+	cmd = ndctl_dimm_cmd_new_smart(t_dimm->dimm);
+	if (!cmd) {
+		dbg(ctx, "new smart command error");
+		goto out;
+	}
+
+	rc = ndctl_cmd_submit(cmd);
+	if (rc || ndctl_cmd_get_firmware_status(cmd)) {
+		dbg(ctx, "subbmit smart command error");
+		goto out;
+	}
+
+	flags = ndctl_cmd_smart_get_flags(cmd);
+	if (flags & ND_SMART_HEALTH_VALID) {
+		unsigned int health = ndctl_cmd_smart_get_health(cmd);
+		if (health & ND_SMART_FATAL_HEALTH)
+			strcat(msg, "fatal");
+		else if (health & ND_SMART_CRITICAL_HEALTH)
+			strcat(msg, "critical");
+		else if (health & ND_SMART_NON_CRITICAL_HEALTH)
+			strcat(msg, "non-critical");
+		else
+			strcat(msg, "ok");
+	} else {
+		strcat(msg, "failed to get data");
+	}
+	strcat(msg, "\n spares_percentage: ");
+
+	if (flags & ND_SMART_SPARES_VALID) {
+		char buf[6];
+		snprintf(buf, 6, "%d", ndctl_cmd_smart_get_spares(cmd));
+		strcat(msg, buf);
+	}
+	else
+		strcat(msg, "failed to get data");
+
+	ndctl_cmd_unref(cmd);
+	return msg;
+
+ out:
+	strcat(msg, "failed to get data\n spares_percentage: failed to get data");
+	return msg;
+}
+
+static void get_notice_msg(struct ndctl_ctx *ctx, struct threshold_dimm *t_dimm,
+				char **log_msg)
+{
+	char *msg;
+	msg = (char *) malloc(2048);
+	msg[0] = '\0';
+	strcat(msg, "nvdimm dimm over threshold notify\n");
+
+	if (strcmp(nvdimmd_cf->logformat, "json") == 0)
+		strcat(msg, get_json_msg(ctx, t_dimm));
+	else
+		strcat(msg, get_text_msg(ctx, t_dimm));
+
+	*log_msg = msg;
+}
+
+int log_notify(struct ndctl_ctx *ctx, struct threshold_dimm *t_dimm,
+		int cnt_dimm, fd_set fds, int cnt_select)
+{
+	int log_notify = 0;
+	char *log_msg;
+
+	for (int i = 0; i < cnt_dimm; i++) {
+		if (log_notify >= cnt_select)
+			break;
+
+		if (!FD_ISSET(t_dimm[i].health_eventfd, &fds))
+			continue;
+		log_notify++;
+
+		get_notice_msg(ctx, &t_dimm[i], &log_msg);
+		notice(ctx, "%s", log_msg);
+		free(log_msg);
+	}
+	return log_notify;
+}
+
+int get_threshold_dimm(struct ndctl_ctx *ctx, struct threshold_dimm *t_dimm,
+			fd_set *fds, int *maxfd)
+{
+	struct ndctl_bus *bus;
+	struct ndctl_dimm *dimm;
+	bool flag = false;
+	int fd, cnt_dimm = 0;
+	char buf[BUF_SIZE];
+
+	ndctl_bus_foreach(ctx, bus) {
+		for (int i = 0; i < cnt_mbus; i++) {
+			if (util_bus_filter(bus, buslist[i])) {
+				flag = true;
+				break;
+			}
+		}
+		if (!flag)
+			continue;
+		flag = false;
+		ndctl_dimm_foreach(bus, dimm) {
+			for (int j = 0; j < cnt_mdimm; j++) {
+				if (util_dimm_filter(dimm, dimmlist[j])) {
+					flag = true;
+					break;
+				}
+			}
+			if (!flag)
+				continue;
+			flag = false;
+			if (!ndctl_dimm_is_cmd_supported(dimm,
+					ND_CMD_SMART_THRESHOLD))
+				continue;
+			t_dimm[cnt_dimm].dimm = dimm;
+			t_dimm[cnt_dimm].devname = ndctl_dimm_get_devname(dimm);
+			fd = ndctl_dimm_get_health_eventfd(dimm);
+			read(fd, buf, sizeof(buf));
+			t_dimm[cnt_dimm].health_eventfd = fd;
+
+			if (fds)
+				FD_SET(fd, fds);
+			if (maxfd) {
+				if (*maxfd < fd)
+					*maxfd = fd;
+			}
+			cnt_dimm++;
+		}
+	}
+	return cnt_dimm;
+}
+
+int get_nvdimmd_conf(const char *path)
+{
+	FILE *f;
+	char buf[BUF_SIZE];
+	struct nvdimmd_conf *conf;
+
+	f = fopen(path, "r");
+	if (f == NULL) {
+		syslog(LOG_ERR, "%s cannot be found\n", path);
+		return -1;
+	}
+
+	conf = calloc(1, sizeof(struct nvdimmd_conf));
+	if (conf == NULL)
+		goto file_err;
+
+	while (fgets(buf, BUF_SIZE, f) != NULL) {
+		char *key;
+		char *value;
+
+		key = trim_string(buf);
+
+		if (key[0] == '#' || key[0] == '\0'){
+			continue;
+		}
+
+		value = strchr(key, '=');
+		if (value == NULL)
+			continue;
+
+		value[0] = '\0';
+		value++;
+
+		key = trim_string(key);
+		if (key == NULL)
+			continue;
+		value = trim_string(value);
+		if (value == NULL)
+			continue;
+
+		if (strcmp(key, "LOGFILE") == 0) {
+			set_string(&conf->logfile, value);
+			continue;
+		}
+		if (strcmp(key, "LOGFORMAT") == 0) {
+			set_string(&conf->logformat, value);
+			continue;
+		}
+		if (strcmp(key, "MONITORBUS") == 0) {
+			set_string(&conf->mbus, value);
+			continue;
+		}
+		if (strcmp(key, "MONITORDIMM") == 0) {
+			set_string(&conf->mdimm, value);
+			continue;
+		}
+	}
+	fclose(f);
+
+	if (conf->logfile == NULL || conf->logformat == NULL
+		|| conf->mbus == NULL || conf->mdimm == NULL)
+		return -1;
+
+	nvdimmd_cf = conf;
+	cnt_mbus = split_string(nvdimmd_cf->mbus, ",", buslist);
+	cnt_mdimm = split_string(nvdimmd_cf->mdimm, ",", dimmlist);
+	return 0;
+
+file_err:
+	fclose(f);
+	return -1;
+}
nvdimmd/libnvdimmd.h new file +53 -0

53 lines not on this page · raw message

Signature · 7 lines
2.9.5


_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
author
#9 lore raw

RFC PATCH v2 4/7 ndctl: nvdimmd: add body file of nvdimm daemon

In reply to QI Fuli

This patch adds the body file of nvdimm daemon and compiles nvdimmd with automake.

---
 Makefile.am         |   2 +-
 configure.ac        |   1 +
 nvdimmd/Makefile.am |  19 +++++++++
 nvdimmd/nvdimmd.c   | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 133 insertions(+), 1 deletion(-)
 create mode 100644 nvdimmd/Makefile.am
 create mode 100644 nvdimmd/nvdimmd.c
Makefile.am +1 -1
index b538b1f..ff91439 100644
@@ -1,7 +1,7 @@
 include Makefile.am.in
 
 ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS}
-SUBDIRS = . daxctl/lib ndctl/lib ndctl daxctl
+SUBDIRS = . daxctl/lib ndctl/lib ndctl daxctl nvdimmd
 if ENABLE_DOCS
 SUBDIRS += Documentation/ndctl Documentation/daxctl
 endif
configure.ac +1 -0
index 5b10381..3ce71e3 100644
@@ -274,6 +274,7 @@ AC_CONFIG_FILES([
         daxctl/lib/Makefile
         ndctl/lib/Makefile
         ndctl/Makefile
+	nvdimmd/Makefile
         daxctl/Makefile
         test/Makefile
         Documentation/ndctl/Makefile
nvdimmd/Makefile.am new file +19 -0
new file mode 100644
index 0000000..9883edd
@@ -0,0 +1,19 @@
+include $(top_srcdir)/Makefile.am.in
+
+bin_PROGRAMS = nvdimmd
+
+nvdimmd_SOURCES =\
+		nvdimmd.c \
+		libnvdimmd.c \
+		util.c \
+		../util/log.c \
+		../util/json.c \
+		../ndctl/util/json-smart.c
+
+nvdimmd_LDADD = ../ndctl/lib/libndctl.la \
+		../daxctl/lib/libdaxctl.la \
+		../libutil.a \
+		$(KMOD_LIBS) \
+		$(JSON_LIBS) \
+		$(UUID_LIBS) \
+		$(UDEV_LIBS)
nvdimmd/nvdimmd.c new file +112 -0

112 lines not on this page · raw message

Signature · 7 lines
2.9.5


_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
#10 lore raw

Re: [RFC PATCH v2 4/7] ndctl: nvdimmd: add body file of nvdimm daemon

Quoted 163 lines · QI Fuli

This patch adds the body file of nvdimm daemon and compiles nvdimmd with automake.

---
 Makefile.am         |   2 +-
 configure.ac        |   1 +
 nvdimmd/Makefile.am |  19 +++++++++
 nvdimmd/nvdimmd.c   | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 133 insertions(+), 1 deletion(-)
 create mode 100644 nvdimmd/Makefile.am
 create mode 100644 nvdimmd/nvdimmd.c
Makefile.am +1 -1
index b538b1f..ff91439 100644
@@ -1,7 +1,7 @@
 include Makefile.am.in
 
 ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS}
-SUBDIRS = . daxctl/lib ndctl/lib ndctl daxctl
+SUBDIRS = . daxctl/lib ndctl/lib ndctl daxctl nvdimmd
 if ENABLE_DOCS
 SUBDIRS += Documentation/ndctl Documentation/daxctl
 endif
configure.ac +1 -0
index 5b10381..3ce71e3 100644
@@ -274,6 +274,7 @@ AC_CONFIG_FILES([
         daxctl/lib/Makefile
         ndctl/lib/Makefile
         ndctl/Makefile
+       nvdimmd/Makefile
         daxctl/Makefile
         test/Makefile
         Documentation/ndctl/Makefile
nvdimmd/Makefile.am new file +19 -0
new file mode 100644
index 0000000..9883edd
@@ -0,0 +1,19 @@
+include $(top_srcdir)/Makefile.am.in
+
+bin_PROGRAMS = nvdimmd
+
+nvdimmd_SOURCES =\
+               nvdimmd.c \
+               libnvdimmd.c \
+               util.c \
+               ../util/log.c \
+               ../util/json.c \
+               ../ndctl/util/json-smart.c
+
+nvdimmd_LDADD = ../ndctl/lib/libndctl.la \
+               ../daxctl/lib/libdaxctl.la \
+               ../libutil.a \
+               $(KMOD_LIBS) \
+               $(JSON_LIBS) \
+               $(UUID_LIBS) \
+               $(UDEV_LIBS)
nvdimmd/nvdimmd.c new file +94 -0
new file mode 100644
index 0000000..f451751
@@ -0,0 +1,112 @@
+/*
+ * Copyright (c) 2017, FUJITSU LIMITED. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU Lesser General Public License,
+ * version 2.1, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT ANY
+ * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for
+ * more details.
+ */
+
+/*
+ * Nvdimm daemon is used to monitor the features of over threshold events.
+ * It automatically searches and monitors all of the dimms which support smart
+ * threshold. When an over threshold event fires, it will write a notification
+ * into the system log.
+ */
+
+#include <stdlib.h>
+#include <unistd.h>
+#include <string.h>
+#include <signal.h>
+#include <sys/stat.h>
+#include <syslog.h>
+#include <errno.h>
+#include <ndctl/lib/private.h>
+#include "libnvdimmd.h"
+
+static int wait_threshold_notify(struct ndctl_ctx *ctx)
+{
+       int rc, maxfd, cnt_dimm;
+       struct threshold_dimm *t_dimm;
+
+       t_dimm = calloc(NUM_MAX_DIMM, sizeof(struct threshold_dimm));
+       if (!t_dimm) {
+               err(ctx, "t_dimm memory not allocated");
+               goto out;
+       }
+
+       fd_set fds;
+       FD_ZERO(&fds);
+
+       cnt_dimm = get_threshold_dimm(ctx, t_dimm, &fds, &maxfd);
+       if (cnt_dimm == 0) {
+               err(ctx, "no dimm supports over threshold notification\n");
+               goto out;
+       }
+
+       rc = select(maxfd + 1, NULL, NULL, &fds, NULL);
+       if (rc < 1) {
+               if (rc == 0)
+                       err(ctx, "select unexpected timeout\n");
+               else
+                       err(ctx, "select %s\n", strerror(errno));
+               goto out;
+       }
+
+       log_notify(ctx, t_dimm, cnt_dimm, fds, rc);
+
+       free(t_dimm);
+       return 0;
+
+out:
+       return 1;
+}
+
+static void nvdimmd_log_init(struct ndctl_ctx *ctx, int priority,
+                               const char *owner, const char *log_env)
+{
+       log_init(&ctx->ctx, owner, log_env);
+       ndctl_set_log_priority(ctx, priority);
+       if (strcmp(nvdimmd_cf->logfile, "syslog") == 0)
+               ndctl_set_log_fn(ctx, log_syslog);
+       else
+               ndctl_set_log_fn(ctx, log_file);
+}
+
+int main(void)
+{
+       if (daemon(0, 0) != 0) {
+               syslog(LOG_ERR, "nvdimmd error: daemon start failed\n");
+               exit(EXIT_FAILURE);
+       }
+       struct ndctl_ctx *ctx;
+       int rc, ret = 0;
+       const char *conf_path = "/etc/nvdimmd/nvdimmd.conf";
+
+       syslog(LOG_INFO, "nvdimmd: started\n");
+
+       if (get_nvdimmd_conf(conf_path) != 0) {
+               syslog(LOG_ERR, "nvdimmd.conf error \n");
+               goto out;

nvdimmd/nvdimmd.c line 104

Quoted 10 lines
nvdimmd/nvdimmd.c new file +10 -0
+       }
+
+       while (ret == 0) {
+               rc = ndctl_new(&ctx);
+               if (rc)
+                       goto out;
+
+               nvdimmd_log_init(ctx, LOG_NOTICE, "nvdimmd", "NVDIMMD_LOG");
+               ret = wait_threshold_notify(ctx);
+               ndctl_unref(ctx);

We shouldn't need to setup and teardown the ndctl_ctx after each wake up.

In general I think we need to start with the end result of what data and events do want to emit and have in the logs and then work backwards from there. I think the first step is define a set of event names and event record details.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

author
#11 lore raw

RFC PATCH v2 5/7 ndctl: nvdimmd: add nvdimmd config file

In reply to QI Fuli

This patch adds nvdimmd.conf.
Autotools to install the config file for nvdimmd service.
The nvdimmd.conf includs the following parameters.
- LOGFILE: Users can choose either to write the notifications into syslog (LOGFILE=syslog), or to send into a special file under /var/log/nvdimmd/ (such as LOGFILE=nvdimmd.log) by configuring LOGFILE parameter in the config file.
- LOGFORMAT: Users can choose the format of notification to be structured json(LOGFORMAT=json) or text(LOGFORMAT=text) by configuring LOGFORMAT parameter in the config file.
- MONITORDIMM: The dimms to monitor are filtered via dimm's name by setting MONITORDIMM parameter in the config file. When setting up multiple dimms, the names of dimms should be separated by ",".(MONITORDIMM=nmem0,nmem1,nmem2)
- MONITORBUS: The dimms to monitor are filtered via its parent bus by setting MONITORBUS parameter as well. When setting up multiple dimms, the names of buses should be separated by "," (MONITORBUS=ndbus0,ndbus1)
The nvdimmd.conf would be read when nvdimmd service starts. Users need to restart nvdimmd service after changing the value of paramaters in nvdimmd.conf.

---
 nvdimmd/Makefile.am  |  9 +++++++++
 nvdimmd/nvdimmd.conf | 25 +++++++++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 nvdimmd/nvdimmd.conf
nvdimmd/Makefile.am +9 -0
index 9883edd..6716547 100644
@@ -17,3 +17,12 @@ nvdimmd_LDADD = ../ndctl/lib/libndctl.la \
 		$(JSON_LIBS) \
 		$(UUID_LIBS) \
 		$(UDEV_LIBS)
+
+conffiles = nvdimmd.conf
+
+confdir = /etc/nvdimmd/
+
+conf_DATA = $(conffiles)
+
+EXTRA_DIST =\
+	$(conffiles)
nvdimmd/nvdimmd.conf new file +25 -0
new file mode 100644
index 0000000..4e6a12a
@@ -0,0 +1,25 @@
+# This is the main Nvdimm daemon configuration file. It contains the
+# configuration directives that give nvdimmd its instructions.
+# You can change settings by editing this file.
+# The changed item will work after restart nvdimmd service.
+# In this file, lines starting with a hash (#) are comments.
+# The following charecters are not allowed the value: : # ? / \ % " '
+
+# LOGFILE: Users can choose either to write the notifications into syslog
+# (LOGFILE=syslog), or to send into a special file under /var/log/nvdimmd/
+# (such as LOGFILE=nvdimmd.log) by setting LOGFILE.
+LOGFILE=nvdimmd.log
+
+# LOGFORMAT: Users can choose the format of notification to be json
+# (LOGFORMAT=json) or text(LOGFORMAT=text) by setting LOGFORMAT.
+LOGFORMAT = json
+
+# MONITORDIMM: The dimms to monitor are filtered via dimm's name by setting
+# MONITORDIMM. When setting up mutiple dimms, the names of dimms should be
+# separated by ",".(MONITORDIMM=nmem0,nmem1,nmem2)
+MONITORDIMM=all
+
+# MONITORBUS: The dimms to monitor are filtered via its parent bus by
+# setting as well. When setting up mutiple dimms, the names of buses
+# should be seprated by "," (MONITORBUS=ndbus0,ndbus1)
+MONITORBUS=all
Signature · 7 lines
2.9.5


_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
author
#12 lore raw

RFC PATCH v2 6/7 ndctl: nvdimmd: add the unit file of systemd for nvdimmd service

In reply to QI Fuli

This patch adds and autotools to install the unit file of systemd for nvdimmd service.

---
 nvdimmd/Makefile.am     | 7 +++++++
 nvdimmd/nvdimmd.service | 7 +++++++
 2 files changed, 14 insertions(+)
 create mode 100644 nvdimmd/nvdimmd.service
nvdimmd/Makefile.am +7 -0
index 6716547..bca0bc8 100644
@@ -18,6 +18,12 @@ nvdimmd_LDADD = ../ndctl/lib/libndctl.la \
 		$(UUID_LIBS) \
 		$(UDEV_LIBS)
 
+unitfiles = nvdimmd.service
+
+unitdir = /usr/lib/systemd/system/
+
+unit_DATA = $(unitfiles)
+
 conffiles = nvdimmd.conf
 
 confdir = /etc/nvdimmd/
@@ -25,4 +31,5 @@ confdir = /etc/nvdimmd/
 conf_DATA = $(conffiles)
 
 EXTRA_DIST =\
+	$(unitfiles) \
 	$(conffiles)
nvdimmd/nvdimmd.service new file +7 -0
new file mode 100644
index 0000000..0b96aaa
@@ -0,0 +1,7 @@
+[Unit]
+Description=Nvdimm Daemon
+
+[Service]
+Type=forking
+ExecStart=/usr/bin/nvdimmd
+ExecStop=/usr/bin/kill ${MAINPID}
Signature · 7 lines
2.9.5


_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
author
#13 lore raw

RFC PATCH v2 7/7 ndctl: nvdimmd: add a temporary test for nvdimm daemon

In reply to QI Fuli

This patch is used to test nvdimmd service temporarily. I will make it an ndctl command or an option of ndctl inject-error in next step.
Currently, You can test nvdimm daemon with the following command.

nvdimmd_test [nmemX] [all]

---
 nvdimmd/Makefile.am    |  14 ++++-
 nvdimmd/nvdimmd_test.c | 142 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 155 insertions(+), 1 deletion(-)
 create mode 100644 nvdimmd/nvdimmd_test.c
nvdimmd/Makefile.am +13 -1
index bca0bc8..cca1980 100644
@@ -1,6 +1,6 @@
 include $(top_srcdir)/Makefile.am.in
 
-bin_PROGRAMS = nvdimmd
+bin_PROGRAMS = nvdimmd nvdimmd_test
 
 nvdimmd_SOURCES =\
 		nvdimmd.c \
@@ -18,6 +18,18 @@ nvdimmd_LDADD = ../ndctl/lib/libndctl.la \
 		$(UUID_LIBS) \
 		$(UDEV_LIBS)
 
+nvdimmd_test_SOURCES =\
+		nvdimmd_test.c \
+		../util/log.c
+
+nvdimmd_test_LDADD =\
+		../ndctl/lib/libndctl.la \
+		../daxctl/lib/libdaxctl.la \
+		../libutil.a \
+		$(KMOD_LIBS) \
+		$(UUID_LIBS) \
+		$(UDEV_LIBS)
+
 unitfiles = nvdimmd.service
 
 unitdir = /usr/lib/systemd/system/
nvdimmd/nvdimmd_test.c new file +142 -0

142 lines not on this page · raw message

Signature · 7 lines
2.9.5


_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
#14 lore raw
In reply to QI Fuli
Quoted 39 lines · QI Fuli

Hi, here is my second version of nvdimm daemon, It would be appreciated if you could check it.

Change log since v1:
- Adding a config file(/etc/nvdimmd/nvdimmd.conf)
- Using struct log_ctx instead of syslog()
- Using log_syslog() to save the notify messages to syslog
- Using log_file() to save the notify messages to special file
- Adding LOG_NOTICE level into log_priority
- Using automake instead of Makefile
- Adding a new util file(nvdimmd/util.c) including helper functions needed for nvdimm daemon.
- Adding nvdimmd_test program

---
This is a patch set of nvdimmd, a tiny daemon to monitor the features of over threshold events. When an over thershold event fires, nvdimmd will output the notification including dimm health status to syslog or a special file to users' configuration. Users can choose the output format to be structured json or text.

Here are out put samples.
- json format:
2017/11/10 11:15:03 [28065] log_notify: nvdimm dimm over threshold notify {
"dev":"nmem1",
"id":"cdab-0a-07e0-feffffff",
"handle":1,
"phys_id":1,

  "health":{
    "health_state":"non-critical",
    "temperature_celsius":23,
    "spares_percentage":75,
    "alarm_temperature":true,
    "alarm_spares":true,
    "temperature_threshold":40,
    "spares_threshold":5,
    "life_used_percentage":5,
    "shutdown_state":"clean"
Quoted 7 lines

}
}
- text format:
2017/11/10 16:21:53 [12479] log_notify: nvdimm dimm over threshold notify
dev: nmem1
health_state: non-critical
spares_percentage: 75

Are these formats that services like Scribe and Fluent can consume? I'd like to see a sample hook up to those applications.

I'm also wondering that since this will be a long running service that may gain more functionality over time, if we should write it in a language that does not have the security sharp edges of C.

I've been looking for an excuse to use the Go language on a project, and this seems a good opportunity.

Quoted 3 lines

TODO list:
- The dimms to monitor should be filtered by namespace and region

Yes, I'd like to make this equivalent to 'ndctl list' in terms filtering.

Quoted 2 lines

- Add more information into the notify message
- Make nvdimmd_test an ndctl command or an option of ndctl inject-error

Yes, the test events should originate from "ndctl inject-error".
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#15 lore raw
Quoted 51 lines · QI Fuli
Quoted 39 lines

Hi, here is my second version of nvdimm daemon, It would be appreciated if you could check it.

Change log since v1:
- Adding a config file(/etc/nvdimmd/nvdimmd.conf)
- Using struct log_ctx instead of syslog()
- Using log_syslog() to save the notify messages to syslog
- Using log_file() to save the notify messages to special file
- Adding LOG_NOTICE level into log_priority
- Using automake instead of Makefile
- Adding a new util file(nvdimmd/util.c) including helper functions needed for nvdimm daemon.
- Adding nvdimmd_test program

---
This is a patch set of nvdimmd, a tiny daemon to monitor the features of over threshold events. When an over thershold event fires, nvdimmd will output the notification including dimm health status to syslog or a special file to users' configuration. Users can choose the output format to be structured json or text.

Here are out put samples.
- json format:
2017/11/10 11:15:03 [28065] log_notify: nvdimm dimm over threshold notify {
"dev":"nmem1",
"id":"cdab-0a-07e0-feffffff",
"handle":1,
"phys_id":1,

  "health":{
    "health_state":"non-critical",
    "temperature_celsius":23,
    "spares_percentage":75,
    "alarm_temperature":true,
    "alarm_spares":true,
    "temperature_threshold":40,
    "spares_threshold":5,
    "life_used_percentage":5,
    "shutdown_state":"clean"
Quoted 7 lines

}
}
- text format:
2017/11/10 16:21:53 [12479] log_notify: nvdimm dimm over threshold notify
dev: nmem1
health_state: non-critical
spares_percentage: 75

Are these formats that services like Scribe and Fluent can consume? I'd like to see a sample hook up to those applications.

Quoted 6 lines

I'm also wondering that since this will be a long running service that may gain more functionality over time, if we should write it in a language that does not have the security sharp edges of C.

I've been looking for an excuse to use the Go language on a project, and this seems a good opportunity.

Hmmmmm, This seems to be big change of direction of nvdimmd.

Certainly, go-lang is good and interesting language in my impression. But, if nvdimmd is re-created by go-lang, I suppose nvdimmd will need to discard many functions of ndctl libraries, and need to re-create them by golang.

(Though golang may be able to link C-lang libraries, then its security
will be simlar with C-lang level, not go-lang level, I think.)

Do you intent that we should re-create many functions for golang?

In addition, do you include source code of go-lang nvdimmd in ndctl source code?

Thanks,

Quoted 7 lines
Quoted 4 lines

TODO list:
- The dimms to monitor should be filtered by namespace and region

Yes, I'd like to make this equivalent to 'ndctl list' in terms filtering.

Quoted 8 lines
Quoted 2 lines

- Add more information into the notify message
- Make nvdimmd_test an ndctl command or an option of ndctl inject-error

Yes, the test events should originate from "ndctl inject-error".
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#16 lore raw
Quoted 64 lines · Yasunori Goto
Quoted 51 lines · QI Fuli
Quoted 39 lines

Hi, here is my second version of nvdimm daemon, It would be appreciated if you could check it.

Change log since v1:
- Adding a config file(/etc/nvdimmd/nvdimmd.conf)
- Using struct log_ctx instead of syslog()
- Using log_syslog() to save the notify messages to syslog
- Using log_file() to save the notify messages to special file
- Adding LOG_NOTICE level into log_priority
- Using automake instead of Makefile
- Adding a new util file(nvdimmd/util.c) including helper functions needed for nvdimm daemon.
- Adding nvdimmd_test program

---
This is a patch set of nvdimmd, a tiny daemon to monitor the features of over threshold events. When an over thershold event fires, nvdimmd will output the notification including dimm health status to syslog or a special file to users' configuration. Users can choose the output format to be structured json or text.

Here are out put samples.
- json format:
2017/11/10 11:15:03 [28065] log_notify: nvdimm dimm over threshold notify {
"dev":"nmem1",
"id":"cdab-0a-07e0-feffffff",
"handle":1,
"phys_id":1,

  "health":{
    "health_state":"non-critical",
    "temperature_celsius":23,
    "spares_percentage":75,
    "alarm_temperature":true,
    "alarm_spares":true,
    "temperature_threshold":40,
    "spares_threshold":5,
    "life_used_percentage":5,
    "shutdown_state":"clean"
Quoted 7 lines

}
}
- text format:
2017/11/10 16:21:53 [12479] log_notify: nvdimm dimm over threshold notify
dev: nmem1
health_state: non-critical
spares_percentage: 75

Are these formats that services like Scribe and Fluent can consume? I'd like to see a sample hook up to those applications.

Quoted 6 lines

I'm also wondering that since this will be a long running service that may gain more functionality over time, if we should write it in a language that does not have the security sharp edges of C.

I've been looking for an excuse to use the Go language on a project, and this seems a good opportunity.

Hmmmmm, This seems to be big change of direction of nvdimmd.

Certainly, go-lang is good and interesting language in my impression. But, if nvdimmd is re-created by go-lang, I suppose nvdimmd will need to discard many functions of ndctl libraries, and need to re-create them by golang.

Quoted 6 lines

(Though golang may be able to link C-lang libraries, then its security
will be simlar with C-lang level, not go-lang level, I think.)

Do you intent that we should re-create many functions for golang?

In addition, do you include source code of go-lang nvdimmd in ndctl source code?

Yes, but let's continue with C for now, I took a look and the impact is not quite as straightforward as I thought.

The next step is to build a demonstration of Scribe or Fluentd consuming nvdimmd events.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

author
#17 lore raw
Quoted 66 lines · Dan Williams
Quoted 65 lines · Yasunori Goto
Quoted 50 lines · QI Fuli
Quoted 41 lines

Hi, here is my second version of nvdimm daemon, It would be appreciated if you could check it.

Change log since v1:
- Adding a config file(/etc/nvdimmd/nvdimmd.conf)
- Using struct log_ctx instead of syslog()
- Using log_syslog() to save the notify messages to syslog
- Using log_file() to save the notify messages to special file
- Adding LOG_NOTICE level into log_priority
- Using automake instead of Makefile
- Adding a new util file(nvdimmd/util.c) including helper functions needed for nvdimm daemon.
- Adding nvdimmd_test program

---
This is a patch set of nvdimmd, a tiny daemon to monitor the features of over threshold events. When an over thershold event fires, nvdimmd will output the notification including dimm health status to syslog or a special file to users' configuration. Users can choose the output format to be structured json or text.

Here are out put samples.
- json format:
2017/11/10 11:15:03 [28065] log_notify: nvdimm dimm over threshold notify

{
   "dev":"nmem1",
   "id":"cdab-0a-07e0-feffffff",
   "handle":1,
   "phys_id":1,
   "health":{
     "health_state":"non-critical",
     "temperature_celsius":23,
     "spares_percentage":75,
     "alarm_temperature":true,
     "alarm_spares":true,
     "temperature_threshold":40,
     "spares_threshold":5,
     "life_used_percentage":5,
     "shutdown_state":"clean"
   }
}
Quoted 5 lines

- text format:
2017/11/10 16:21:53 [12479] log_notify: nvdimm dimm over threshold notify
dev: nmem1
health_state: non-critical
spares_percentage: 75

Are these formats that services like Scribe and Fluent can consume? I'd like to see a sample hook up to those applications.

Quoted 6 lines

I'm also wondering that since this will be a long running service that may gain more functionality over time, if we should write it in a language that does not have the security sharp edges of C.

I've been looking for an excuse to use the Go language on a project, and this seems a good opportunity.

Hmmmmm, This seems to be big change of direction of nvdimmd.

Certainly, go-lang is good and interesting language in my impression. But, if nvdimmd is re-created by go-lang, I suppose nvdimmd will need to discard many functions of ndctl libraries, and need to re-create them by golang.

(Though golang may be able to link C-lang libraries, then its security
will be simlar with C-lang level, not go-lang level, I think.)

Quoted 9 lines
Quoted 3 lines

Do you intent that we should re-create many functions for golang?

In addition, do you include source code of go-lang nvdimmd in ndctl source code?

Yes, but let's continue with C for now, I took a look and the impact is not quite as straightforward as I thought.

The next step is to build a demonstration of Scribe or Fluentd consuming nvdimmd events.

Ok, I will make a sample hook up to Fluentd.

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

author
#18 lore raw
Quoted 71 lines · Fuli
Quoted 68 lines · Dan Williams
Quoted 67 lines · Yasunori Goto
Quoted 50 lines · QI Fuli
Quoted 38 lines

Hi, here is my second version of nvdimm daemon, It would be appreciated if you could check it.

Change log since v1:
  - Adding a config file(/etc/nvdimmd/nvdimmd.conf)
  - Using struct log_ctx instead of syslog()
    - Using log_syslog() to save the notify messages to syslog
    - Using log_file() to save the notify messages to special file
  - Adding LOG_NOTICE level into log_priority
  - Using automake instead of Makefile
  - Adding a new util file(nvdimmd/util.c) including helper functions needed for nvdimm daemon.
  - Adding nvdimmd_test program

---
This is a patch set of nvdimmd, a tiny daemon to monitor the features of over threshold events. When an over thershold event fires, nvdimmd will output the notification including dimm health status to syslog or a special file to users' configuration. Users can choose the output format to be structured json or text.

Here are out put samples.
- json format:
2017/11/10 11:15:03 [28065] log_notify: nvdimm dimm over threshold notify {
   "dev":"nmem1",
   "id":"cdab-0a-07e0-feffffff",
   "handle":1,
   "phys_id":1,
   "health":{
     "health_state":"non-critical",
     "temperature_celsius":23,
     "spares_percentage":75,
     "alarm_temperature":true,
     "alarm_spares":true,
     "temperature_threshold":40,
     "spares_threshold":5,
     "life_used_percentage":5,

Quoted 8 lines

     "shutdown_state":"clean"
   }
}
- text format:
2017/11/10 16:21:53 [12479] log_notify: nvdimm dimm over threshold notify
dev: nmem1
  health_state: non-critical
  spares_percentage: 75

Are these formats that services like Scribe and Fluent can consume? I'd like to see a sample hook up to those applications.

Quoted 6 lines

I'm also wondering that since this will be a long running service that may gain more functionality over time, if we should write it in a language that does not have the security sharp edges of C.

I've been looking for an excuse to use the Go language on a project, and this seems a good opportunity.

Hmmmmm, This seems to be big change of direction of nvdimmd.

Certainly, go-lang is good and interesting language in my impression. But, if nvdimmd is re-created by go-lang, I suppose nvdimmd will need to discard many functions of ndctl libraries, and need to re-create them by golang.

(Though golang may be able to link C-lang libraries, then its security will be simlar with C-lang level, not go-lang level, I think.)

Do you intent that we should re-create many functions for golang?

Quoted 10 lines
Quoted 7 lines
Quoted 1 line

In addition, do you include source code of go-lang nvdimmd in ndctl source code?

Yes, but let's continue with C for now, I took a look and the impact is not quite as straightforward as I thought.

The next step is to build a demonstration of Scribe or Fluentd consuming nvdimmd events.

Ok, I will make a sample hook up to Fluentd.

I tried to hook up the current json format output to Fluentd, Fluentd parsed each line in the output as a new event.
The current json format output cannot be consumed by Fluentd without any plugins. This is undesired.
So I changed the multiline json format output to one-line like the following sample, then Fluentd parsed it well.

{"datetime":"2017/11/28 11:00:12","pid":7013,"dev":"nmem1","health":{"health_state":"non-critical","temperature_celsius":23,"spares_percentage":75","life_used_percentage":5,"shutdown_state":"clean"}}

I think one-line output will get consumed more easily by other data collect services besides Fluentd. All the output items should be placed in json block, such as datetime, eventid.

So I would like to change current multi-line json format output to one line and move all items into json block.

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#19 lore raw

On Thu, Dec 7, 2017 at 1:58 AM, Qi, Fuli <[email protected]> wrote:
[..]

Quoted 10 lines
Quoted 3 lines

Ok, I will make a sample hook up to Fluentd.

I tried to hook up the current json format output to Fluentd, Fluentd parsed each line in the output as a new event.
The current json format output cannot be consumed by Fluentd without any plugins. This is undesired.
So I changed the multiline json format output to one-line like the following sample, then Fluentd parsed it well.

Quoted 9 lines

{"datetime":"2017/11/28 11:00:12","pid":7013,"dev":"nmem1","health":{"health_state":"non-critical","temperature_celsius":23,"spares_percentage":75","life_used_percentage":5,"shutdown_state":"clean"}}

I think one-line output will get consumed more easily by other data collect services besides Fluentd. All the output items should be placed in json block, such as datetime, eventid.

So I would like to change current multi-line json format output to one line and move all items into json block.

This is just a small matter of using the JSON_C_TO_STRING_PLAIN flag instead of JSON_C_TO_STRING_PRETTY when emitting json (json_object_to_json_string_ext()) for Fluentd or another service to consume.

Also, when creating a json record for Fluentd I think the standard record that ndctl list reports should be wrapped as a sub-object i.e:

{
"dev":"nmem3",
"id":"cdab-0a-07e0-feffffff",
"handle":1,
"phys_id":1,

  "health":{
    "health_state":"non-critical",
    "temperature_celsius":23,
    "spares_percentage":75,
    "alarm_temperature":true,
    "alarm_spares":true,
    "temperature_threshold":80.125,
    "spares_threshold":128,
    "life_used_percentage":5,
    "shutdown_state":"clean"

}
}

...becomes:

{
"timestamp":"YYYY/MM/DD HH:mm:ss",
"pid":1234,

  {
    "dev":"nmem3",
    "id":"cdab-0a-07e0-feffffff",
    "handle":1,
    "phys_id":1,
    "health":{
      "health_state":"non-critical",
      "temperature_celsius":23,
      "spares_percentage":75,
      "alarm_temperature":true,
      "alarm_spares":true,
      "temperature_threshold":80.125,
      "spares_threshold":128,
      "life_used_percentage":5,
      "shutdown_state":"clean"
    }

}
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm