Redfish EventService Implementation
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either the phosphor::logging::entry or sd_journal_send() command.
After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
Rajes
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
Rajes
Quoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
Rajes
I have implemented the skeleton code and uploaded it for review. Kindly review and let me know your comments. For prototyping, based on the Redfish Event Logging design, I have modified phosphosr-user-manager application to log resource creation event when an account is created. For reading the redfish event logs from the journal and writing to BMC filesystem(/var/log/redfish), I have pulled the rsyslog configuration from the OEM (Intel) and made minor changes.
These patches are work in progress and needs lot more changes. Any suggestions regarding the approach is appreciated.
https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/29464
https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-user-manager/+/29463
https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/29462
Thanks,
Rajes
Quoted 19 lines · RAJESWARAN THILLAIGOVINDAN
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
RajesQuoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
Rajes
Hi,
I am sorry for late response as this mail is buried under and got struck back of my mind.
As i did mentioned in EventService design document, EventLog Monitor service is not common across the organizations( Example: Intel uses the redfish event logs file and inotify mechanism for monitoring the event logs. Where as IBM uses d-bus event log mechanism and they can relay on match rules). That said challenges with ResourceType mapping will be different in both monitoring mechanisms. This is good point. Initially when i started EventService design, i thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) but not sure that may become difficult when we expand supported ResourceTypes.
As per my reading from below query, You are looking at d-bus match rules and ResourceTypes mapping which is more specific to d-bus event logging(IBM way of implementing event logging). reading it from journal logs will give more information but that will impact the performance to large extent. This might be one of the reason why we (Intel) uses Redfish message ID while logging redfish events logs to journal(You can refer design document for same at https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). In opinion, in your d-bus if you are using some kind of filter(Example REDFISH_MESSAGE_ID) while logging in journal logs for all events and figure out the way to monitor the journal logs without impacting the performance, that should be ok as long as match filters are satisfied for Redfish EventService subscriptions and supported Types(Again differs with implementation).
Thanks,
-Appu
Quoted 19 lines · RAJESWARAN THILLAIGOVINDAN
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
RajesQuoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
Rajes
Hi Apparao,
Quoted 5 lines · Apparao
Hi,
I am sorry for late response as this mail is buried under and got struck back of my mind.
As i did mentioned in EventService design document, EventLog Monitor service is not common across the organizations( Example: Intel uses the redfish event logs file and inotify mechanism for monitoring the event logs. Where as IBM uses d-bus event log mechanism and they can relay on match rules). That said challenges with ResourceType mapping will be different in both monitoring mechanisms. This is good point. Initially when i started EventService design, i thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) but not sure that may become difficult when we expand supported ResourceTypes.
If I am getting correctly, Here is the flow which Intel uses.
1. Individual repo have to push the logs using sd_journal_send which will write to the file(/var/log/redfish) by using rsyslog daemon
sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
"REDFISH_MESSAGE_ID=%s", "ResourceEvent.1.0.ResourceCreated",NULL);
* How you would populate the "OriginOfCondition" during sending of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
* Any plan to add resource path in this journal message which tells that this is the path for which resource created event generated.
* Would the path be "Redfish Path/ D-bus Path"?
* Where the mapping would be done between D-busPath/Redfish Resource path?
Cons: Every application have to make the change(use sd_journal_send). My thought is backend application should not be aware of the redfish terminlogy.
*2.* Some application(bmcweb) would do the Inotify on the path(/var/log/redfish) and send the event once there is any activity on this file.
Quoted 1 line
I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
per Intel design)
Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
Quoted 27 lines
As per my reading from below query, You are looking at d-bus match rules and ResourceTypes mapping which is more specific to d-bus event logging(IBM way of implementing event logging). reading it from journal logs will give more information but that will impact the performance to large extent. This might be one of the reason why we (Intel) uses Redfish message ID while logging redfish events logs to journal(You can refer design document for same at https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). In opinion, in your d-bus if you are using some kind of
filter(Example REDFISH_MESSAGE_ID)while logging in journal logs for all events and figure out the way to monitor the journal logs without impacting the performance, that should be ok as long as match filters are satisfied for Redfish EventService subscriptions and supported Types(Again differs with implementation).Thanks,
-Appu
Quoted 19 lines · RAJESWARAN THILLAIGOVINDAN
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
RajesQuoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
Rajes
Thanks
Ratan
Hi Ratan,
Comments in-line
Quoted 15 lines · Ratan Gupta
Hi Apparao,
Quoted 5 lines · Apparao
Hi,
I am sorry for late response as this mail is buried under and got struck back of my mind.
As i did mentioned in EventService design document, EventLog Monitor service is not common across the organizations( Example: Intel uses the redfish event logs file and inotify mechanism for monitoring the event logs. Where as IBM uses d-bus event log mechanism and they can relay on match rules). That said challenges with ResourceType mapping will be different in both monitoring mechanisms. This is good point. Initially when i started EventService design, i thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) but not sure that may become difficult when we expand supported ResourceTypes.
If I am getting correctly, Here is the flow which Intel uses.
1. Individual repo have to push the logs using
sd_journal_sendwhich will write to the file(/var/log/redfish) by using rsyslog daemon
Quoted 7 lines
sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <
LOG_LEVEL>,"REDFISH_MESSAGE_ID=%s", "ResourceEvent.1.0.ResourceCreated",NULL); * How you would populate the "OriginOfCondition" during sending of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
Currently in logServices( logEntry), we are not reporting the "OriginOfCondition" property as per schema. I will check with Jason( Who wrote the logService) and get back on this.
BTW can you give me how this information is fetched in IBM way of LogService implementation( D-Bus)? If you already ratified any such, i think we can leverage. If not, We work together for solution.
Quoted 4 lines
* Any plan to add resource path in this journal message which tells that this is the path for which resource created event generated.
Same as above.
Quoted 3 lines
* Would the path be "Redfish Path/ D-bus Path"?
As per Redfish specification, This should be "@odata.id" which means it should be of resource uri and so we can't use d-bus path here for OriginOfConditions.
Quoted 7 lines
* Where the mapping would be done between D-busPath/Redfish Resource path?
Cons: Every application have to make the
change(use sd_journal_send). My thought is backend application should not be aware of the redfish terminlogy.
Having separate process only for this mapping may not be good( No different from maintaining that map inside bmcweb as there won't be any other consumers). Ideal way is, that should be mapped while logging logEntry's itself. But we are not doing it currently which need to be re-looked. Give me some time, I will think and check with other folks and get back.
Quoted 7 lines
*2.* Some application(bmcweb) would do the Inotify on the path(/var/log/redfish) and send the event once there is any activity on this file.
Quoted 1 line
I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
per Intel design)
Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
Initially i thought "ResourceType" based event filtering can be done using minimal mapping( Using MessageID and args). But that will work for minimal set. If the supported ResourceTypes grows, we will have bigger challenges which i can sense it now. Anyway, Supported Resources are completely implementation specific. If this value is empty means, by default all event logs will be sent to subscribers. This is what we can start with before supported ResourceTypes list grows.
Quoted 31 lines
Quoted 27 lines
As per my reading from below query, You are looking at d-bus match rules and ResourceTypes mapping which is more specific to d-bus event logging(IBM way of implementing event logging). reading it from journal logs will give more information but that will impact the performance to large extent. This might be one of the reason why we (Intel) uses Redfish message ID while logging redfish events logs to journal(You can refer design document for same at https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). In opinion, in your d-bus if you are using some kind of
filter(Example REDFISH_MESSAGE_ID)while logging in journal logs for all events and figure out the way to monitor the journal logs without impacting the performance, that should be ok as long as match filters are satisfied for Redfish EventService subscriptions and supported Types(Again differs with implementation).Thanks,
-Appu
Quoted 19 lines · RAJESWARAN THILLAIGOVINDAN
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
RajesQuoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
RajesThanks
Ratan
Further to my previous mail, I wanted to see how a JSON dictionary would look like for implementing Redfish EventService based on D-Bus match. The D-Bus match approach is good for implementing life cycle events((ResourceAdded, ResourceRemoved, ResourceUpdated). In bmcweb, a Redfish Resource is viewed as having a set of properties which comes from one or more D-Bus objects. So, when a client subscribes for a ResourceType, using a JSON dictionary, find the D-Bus object(s) mapped to the ResourceType and create matches. When the D-Bus match occurs, again using aJSON dictionary, find the Redfish ResourceType mapped to the D-Bus object and create events. The JSON dictionary should also provide the Redifsh URI which needs to be specified in the event. An example JSON dictionary for mapping the Redifsh ResourceTypes LogEntry and ComputerSystem is available here : https://gist.github.com/trajeswaran/fec230abd36181f85d2f20d09164ec05. In case of LogEntry, there is one to one mapping of ResourceType to D-Bus object. In case of ComputerSystem, the properties comes from multiple D-Bus objects. Do you see any drawback with this approach? Kindly let me know what you think. Thanks in advance.
Quoted 89 lines · Apparao
Hi Ratan,
Comments in-line
Quoted 15 lines · Ratan Gupta
Hi Apparao,
Quoted 5 lines · Apparao
Hi,
I am sorry for late response as this mail is buried under and got struck back of my mind.
As i did mentioned in EventService design document, EventLog Monitor service is not common across the organizations( Example: Intel uses the redfish event logs file and inotify mechanism for monitoring the event logs. Where as IBM uses d-bus event log mechanism and they can relay on match rules). That said challenges with ResourceType mapping will be different in both monitoring mechanisms. This is good point. Initially when i started EventService design, i thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) but not sure that may become difficult when we expand supported ResourceTypes.
If I am getting correctly, Here is the flow which Intel uses.
1. Individual repo have to push the logs using
sd_journal_sendwhich will write to the file(/var/log/redfish) by using rsyslog daemonQuoted 7 lines
sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <
LOG_LEVEL>,"REDFISH_MESSAGE_ID=%s", "ResourceEvent.1.0.ResourceCreated",NULL); * How you would populate the "OriginOfCondition" during sending of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)Currently in logServices( logEntry), we are not reporting the "OriginOfCondition" property as per schema. I will check with Jason( Who wrote the logService) and get back on this.
BTW can you give me how this information is fetched in IBM way of LogService implementation( D-Bus)? If you already ratified any such, i think we can leverage. If not, We work together for solution.
Quoted 4 lines
* Any plan to add resource path in this journal message which tells that this is the path for which resource created event generated.
Same as above.
Quoted 3 lines
* Would the path be "Redfish Path/ D-bus Path"?
As per Redfish specification, This should be "@odata.id" which means it should be of resource uri and so we can't use d-bus path here for OriginOfConditions.
Quoted 7 lines
* Where the mapping would be done between D-busPath/Redfish Resource path?
Cons: Every application have to make the
change(use sd_journal_send). My thought is backend application should not be aware of the redfish terminlogy.Having separate process only for this mapping may not be good( No different from maintaining that map inside bmcweb as there won't be any other consumers). Ideal way is, that should be mapped while logging logEntry's itself. But we are not doing it currently which need to be re-looked. Give me some time, I will think and check with other folks and get back.
Quoted 6 lines
*2.* Some application(bmcweb) would do the Inotify on the path(/var/log/redfish) and send the event once there is any activity on this file.
Quoted 1 line
I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
per Intel design)
Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
Initially i thought "ResourceType" based event filtering can be done using minimal mapping( Using MessageID and args). But that will work for minimal set. If the supported ResourceTypes grows, we will have bigger challenges which i can sense it now. Anyway, Supported Resources are completely implementation specific. If this value is empty means, by default all event logs will be sent to subscribers. This is what we can start with before supported ResourceTypes list grows.
Quoted 31 lines
Quoted 27 lines
As per my reading from below query, You are looking at d-bus match rules and ResourceTypes mapping which is more specific to d-bus event logging(IBM way of implementing event logging). reading it from journal logs will give more information but that will impact the performance to large extent. This might be one of the reason why we (Intel) uses Redfish message ID while logging redfish events logs to journal(You can refer design document for same at https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). In opinion, in your d-bus if you are using some kind of
filter(Example REDFISH_MESSAGE_ID)while logging in journal logs for all events and figure out the way to monitor the journal logs without impacting the performance, that should be ok as long as match filters are satisfied for Redfish EventService subscriptions and supported Types(Again differs with implementation).Thanks,
-Appu
Quoted 19 lines · RAJESWARAN THILLAIGOVINDAN
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
RajesQuoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
RajesThanks
Ratan
Apparao,
I see that you have uploaded a patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for supporting server sent events. This patch supports event filtering based on registry prefix and/or messageId.
I would like to know if you have any plan to support event filtering based on resource type. If so, I would like to work together for a better solution. Earlier, I have proposed a solution based on D-Bus match using a dictionary. With that approach, the major challenge is to map Redfish resource and properties to D-Bus object and properties respectively. If D-Bus applications are modified to include resource type and origin of condition in the event, then there is no need for a map. But,that brings Redfish terminology to the application. Also, this will become an overhead if an OEM is not interested in Redfish event service.
Thanks,
Rajes
Quoted 89 lines · Apparao
Hi Ratan,
Comments in-line
Quoted 15 lines · Ratan Gupta
Hi Apparao,
Quoted 5 lines · Apparao
Hi,
I am sorry for late response as this mail is buried under and got struck back of my mind.
As i did mentioned in EventService design document, EventLog Monitor service is not common across the organizations( Example: Intel uses the redfish event logs file and inotify mechanism for monitoring the event logs. Where as IBM uses d-bus event log mechanism and they can relay on match rules). That said challenges with ResourceType mapping will be different in both monitoring mechanisms. This is good point. Initially when i started EventService design, i thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) but not sure that may become difficult when we expand supported ResourceTypes.
If I am getting correctly, Here is the flow which Intel uses.
1. Individual repo have to push the logs using
sd_journal_sendwhich will write to the file(/var/log/redfish) by using rsyslog daemonQuoted 7 lines
sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <
LOG_LEVEL>,"REDFISH_MESSAGE_ID=%s", "ResourceEvent.1.0.ResourceCreated",NULL); * How you would populate the "OriginOfCondition" during sending of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)Currently in logServices( logEntry), we are not reporting the "OriginOfCondition" property as per schema. I will check with Jason( Who wrote the logService) and get back on this.
BTW can you give me how this information is fetched in IBM way of LogService implementation( D-Bus)? If you already ratified any such, i think we can leverage. If not, We work together for solution.
Quoted 4 lines
* Any plan to add resource path in this journal message which tells that this is the path for which resource created event generated.
Same as above.
Quoted 3 lines
* Would the path be "Redfish Path/ D-bus Path"?
As per Redfish specification, This should be "@odata.id" which means it should be of resource uri and so we can't use d-bus path here for OriginOfConditions.
Quoted 7 lines
* Where the mapping would be done between D-busPath/Redfish Resource path?
Cons: Every application have to make the
change(use sd_journal_send). My thought is backend application should not be aware of the redfish terminlogy.Having separate process only for this mapping may not be good( No different from maintaining that map inside bmcweb as there won't be any other consumers). Ideal way is, that should be mapped while logging logEntry's itself. But we are not doing it currently which need to be re-looked. Give me some time, I will think and check with other folks and get back.
Quoted 6 lines
*2.* Some application(bmcweb) would do the Inotify on the path(/var/log/redfish) and send the event once there is any activity on this file.
Quoted 1 line
I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
per Intel design)
Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
Initially i thought "ResourceType" based event filtering can be done using minimal mapping( Using MessageID and args). But that will work for minimal set. If the supported ResourceTypes grows, we will have bigger challenges which i can sense it now. Anyway, Supported Resources are completely implementation specific. If this value is empty means, by default all event logs will be sent to subscribers. This is what we can start with before supported ResourceTypes list grows.
Quoted 31 lines
Quoted 27 lines
As per my reading from below query, You are looking at d-bus match rules and ResourceTypes mapping which is more specific to d-bus event logging(IBM way of implementing event logging). reading it from journal logs will give more information but that will impact the performance to large extent. This might be one of the reason why we (Intel) uses Redfish message ID while logging redfish events logs to journal(You can refer design document for same at https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). In opinion, in your d-bus if you are using some kind of
filter(Example REDFISH_MESSAGE_ID)while logging in journal logs for all events and figure out the way to monitor the journal logs without impacting the performance, that should be ok as long as match filters are satisfied for Redfish EventService subscriptions and supported Types(Again differs with implementation).Thanks,
-Appu
Quoted 19 lines · RAJESWARAN THILLAIGOVINDAN
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
RajesQuoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
RajesThanks
Ratan
Hi Rajeswaran,
Thanks for your mail. At the moment, I don't have plans to support the "ResourceTypes", "OriginResources" based filtering. Basically Intel uses file systems based redfish event logs( journalctl -> rsync-> filesystem) and doesn't use D-Bus mechanism like IBM uses. So I am not much familiar with D-Bus logging but some of the suggestions:
1) While logging redfish events over D-Bus itself, it can provide details on ResourceTypes and OriginResource URI/Path.
This is ideal and most efficient way. Since we walked a walked long distance from start, Its hard to modify all the services to uses these 2 new input parameters while logging events( Requires change in almost all repo's)
2) For resourcesTypes: Can have mapping dictionary against all MessageId's. For OriginResources: I believe, event log over D-Bus is already holding the Path. If so, last 3/4 nodes of uri can be taken and mapped against the resources and that can be used in Event filtering. We did used same mechanism in case of telemetry while mapping MetricReportDefinitions to URI.
Hope this helps.
Thanks,
-Appu
Quoted 101 lines · RAJESWARAN THILLAIGOVINDAN
Apparao,
I see that you have uploaded a patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for supporting server sent events. This patch supports event filtering based on registry prefix and/or messageId.
I would like to know if you have any plan to support event filtering based on resource type. If so, I would like to work together for a better solution. Earlier, I have proposed a solution based on D-Bus match using a dictionary. With that approach, the major challenge is to map Redfish resource and properties to D-Bus object and properties respectively. If D-Bus applications are modified to include resource type and origin of condition in the event, then there is no need for a map. But,that brings Redfish terminology to the application. Also, this will become an overhead if an OEM is not interested in Redfish event service.
Thanks,
RajesQuoted 91 lines · Apparao
Hi Ratan,
Comments in-line
Quoted 16 lines · Ratan Gupta
Hi Apparao,
Quoted 5 lines · Apparao
Hi,
I am sorry for late response as this mail is buried under and got struck back of my mind.
As i did mentioned in EventService design document, EventLog Monitor service is not common across the organizations( Example: Intel uses the redfish event logs file and inotify mechanism for monitoring the event logs. Where as IBM uses d-bus event log mechanism and they can relay on match rules). That said challenges with ResourceType mapping will be different in both monitoring mechanisms. This is good point. Initially when i started EventService design, i thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) but not sure that may become difficult when we expand supported ResourceTypes.
If I am getting correctly, Here is the flow which Intel uses.
1. Individual repo have to push the logs using
sd_journal_sendwhich will write to the file(/var/log/redfish) by using rsyslog daemonQuoted 8 lines
sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <
LOG_LEVEL>,"REDFISH_MESSAGE_ID=%s", "ResourceEvent.1.0.ResourceCreated",NULL); * How you would populate the "OriginOfCondition" during sending of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)Currently in logServices( logEntry), we are not reporting the "OriginOfCondition" property as per schema. I will check with Jason( Who wrote the logService) and get back on this.
BTW can you give me how this information is fetched in IBM way of LogService implementation( D-Bus)? If you already ratified any such, i think we can leverage. If not, We work together for solution.
Quoted 4 lines
* Any plan to add resource path in this journal message which tells that this is the path for which resource created event generated.
Same as above.
Quoted 3 lines
* Would the path be "Redfish Path/ D-bus Path"?
As per Redfish specification, This should be "@odata.id" which means it should be of resource uri and so we can't use d-bus path here for OriginOfConditions.
Quoted 7 lines
* Where the mapping would be done between D-busPath/Redfish Resource path?
Cons: Every application have to make the
change(use sd_journal_send). My thought is backend application should not be aware of the redfish terminlogy.Having separate process only for this mapping may not be good( No different from maintaining that map inside bmcweb as there won't be any other consumers). Ideal way is, that should be mapped while logging logEntry's itself. But we are not doing it currently which need to be re-looked. Give me some time, I will think and check with other folks and get back.
Quoted 6 lines
*2.* Some application(bmcweb) would do the Inotify on the path(/var/log/redfish) and send the event once there is any activity on this file.
Quoted 1 line
I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
per Intel design)
Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
Initially i thought "ResourceType" based event filtering can be done using minimal mapping( Using MessageID and args). But that will work for minimal set. If the supported ResourceTypes grows, we will have bigger challenges which i can sense it now. Anyway, Supported Resources are completely implementation specific. If this value is empty means, by default all event logs will be sent to subscribers. This is what we can start with before supported ResourceTypes list grows.
Quoted 31 lines
Quoted 27 lines
As per my reading from below query, You are looking at d-bus match rules and ResourceTypes mapping which is more specific to d-bus event logging(IBM way of implementing event logging). reading it from journal logs will give more information but that will impact the performance to large extent. This might be one of the reason why we (Intel) uses Redfish message ID while logging redfish events logs to journal(You can refer design document for same at https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). In opinion, in your d-bus if you are using some kind of
filter(Example REDFISH_MESSAGE_ID)while logging in journal logs for all events and figure out the way to monitor the journal logs without impacting the performance, that should be ok as long as match filters are satisfied for Redfish EventService subscriptions and supported Types(Again differs with implementation).Thanks,
-Appu
Quoted 19 lines · RAJESWARAN THILLAIGOVINDAN
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
RajesQuoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
RajesThanks
Ratan
Apparao,
Thanks a lot for your suggestions. We lean towards using a dictionary to map Redfish ResourceType to D-Bus objects path and vice versa and then using D-Bus match to generate life cycle events. This way, the changes are limited to bmcweb. The resource type and the origin resource URI will be included in the event log. This requires change in the format of event log file /var/log/redfish. I have commented the same in the server sent event patch that you have uploaded. Kindly see if you can leave the parsing of file to the OEMs. That way, the existing infrastructure can be used by the OEMs to support other filtering mechanisms as defined in the specification.
Thanks,
Rajes
Quoted 120 lines · Apparao
Hi Rajeswaran,
Thanks for your mail. At the moment, I don't have plans to support the "ResourceTypes", "OriginResources" based filtering. Basically Intel uses file systems based redfish event
logs( journalctl -> rsync-> filesystem)and doesn't use D-Bus mechanism like IBM uses. So I am not much familiar with D-Bus logging but some of the suggestions:1) While logging redfish events over D-Bus itself, it can provide details on ResourceTypes and OriginResource URI/Path.
This is ideal and most efficient way. Since we walked a walked long distance from start, Its hard to modify all the services to uses these 2 new input parameters while logging events( Requires change in almost all repo's)
2) For resourcesTypes: Can have mapping dictionary against all MessageId's. For OriginResources: I believe, event log over D-Bus is already holding the Path. If so, last 3/4 nodes of uri can be taken and mapped against the resources and that can be used in Event filtering. We did used same mechanism in case of telemetry while mapping MetricReportDefinitions to URI.
Hope this helps.
Thanks,
-Appu
Quoted 101 lines · RAJESWARAN THILLAIGOVINDAN
Apparao,
I see that you have uploaded a patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for supporting server sent events. This patch supports event filtering based on registry prefix and/or messageId.
I would like to know if you have any plan to support event filtering based on resource type. If so, I would like to work together for a better solution. Earlier, I have proposed a solution based on D-Bus match using a dictionary. With that approach, the major challenge is to map Redfish resource and properties to D-Bus object and properties respectively. If D-Bus applications are modified to include resource type and origin of condition in the event, then there is no need for a map. But,that brings Redfish terminology to the application. Also, this will become an overhead if an OEM is not interested in Redfish event service.
Thanks,
RajesQuoted 91 lines · Apparao
Hi Ratan,
Comments in-line
Quoted 16 lines · Ratan Gupta
Hi Apparao,
Quoted 5 lines · Apparao
Hi,
I am sorry for late response as this mail is buried under and got struck back of my mind.
As i did mentioned in EventService design document, EventLog Monitor service is not common across the organizations( Example: Intel uses the redfish event logs file and inotify mechanism for monitoring the event logs. Where as IBM uses d-bus event log mechanism and they can relay on match rules). That said challenges with ResourceType mapping will be different in both monitoring mechanisms. This is good point. Initially when i started EventService design, i thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) but not sure that may become difficult when we expand supported ResourceTypes.
If I am getting correctly, Here is the flow which Intel uses.
1. Individual repo have to push the logs using
sd_journal_sendwhich will write to the file(/var/log/redfish) by using rsyslog daemonQuoted 8 lines
sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <
LOG_LEVEL>,"REDFISH_MESSAGE_ID=%s", "ResourceEvent.1.0.ResourceCreated",NULL); * How you would populate the "OriginOfCondition" during sending of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)Currently in logServices( logEntry), we are not reporting the "OriginOfCondition" property as per schema. I will check with Jason( Who wrote the logService) and get back on this.
BTW can you give me how this information is fetched in IBM way of LogService implementation( D-Bus)? If you already ratified any such, i think we can leverage. If not, We work together for solution.
Quoted 4 lines
* Any plan to add resource path in this journal message which tells that this is the path for which resource created event generated.
Same as above.
Quoted 3 lines
* Would the path be "Redfish Path/ D-bus Path"?
As per Redfish specification, This should be "@odata.id" which means it should be of resource uri and so we can't use d-bus path here for OriginOfConditions.
Quoted 7 lines
* Where the mapping would be done between D-busPath/Redfish Resource path?
Cons: Every application have to make the
change(use sd_journal_send). My thought is backend application should not be aware of the redfish terminlogy.Having separate process only for this mapping may not be good( No different from maintaining that map inside bmcweb as there won't be any other consumers). Ideal way is, that should be mapped while logging logEntry's itself. But we are not doing it currently which need to be re-looked. Give me some time, I will think and check with other folks and get back.
Quoted 6 lines
*2.* Some application(bmcweb) would do the Inotify on the path(/var/log/redfish) and send the event once there is any activity on this file.
Quoted 1 line
I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
per Intel design)
Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
Initially i thought "ResourceType" based event filtering can be done using minimal mapping( Using MessageID and args). But that will work for minimal set. If the supported ResourceTypes grows, we will have bigger challenges which i can sense it now. Anyway, Supported Resources are completely implementation specific. If this value is empty means, by default all event logs will be sent to subscribers. This is what we can start with before supported ResourceTypes list grows.
Quoted 31 lines
Quoted 27 lines
As per my reading from below query, You are looking at d-bus match rules and ResourceTypes mapping which is more specific to d-bus event logging(IBM way of implementing event logging). reading it from journal logs will give more information but that will impact the performance to large extent. This might be one of the reason why we (Intel) uses Redfish message ID while logging redfish events logs to journal(You can refer design document for same at https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). In opinion, in your d-bus if you are using some kind of
filter(Example REDFISH_MESSAGE_ID)while logging in journal logs for all events and figure out the way to monitor the journal logs without impacting the performance, that should be ok as long as match filters are satisfied for Redfish EventService subscriptions and supported Types(Again differs with implementation).Thanks,
-Appu
Quoted 19 lines · RAJESWARAN THILLAIGOVINDAN
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
RajesQuoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
RajesThanks
Ratan
Rajes,
The dictionary to map Redfish resourceType and D-Bus object path( I believe URI intern) is good. It will be great if you share design document, if its done.
At the moment, redfish event logs file(/var/log/redfish) doesn't have ResourceTypes and OriginResource fields. The Existing redfish event log implementation(log service) also doesn't have support for that. You can propose design change for same along with how it is used in event log service. Same thing, can be adopted to EventService, once its agreed by OEM's( I am thinking, it should go under new OEM specific compiler flag. But that we can ratify later).
Thanks,
-Appu
Quoted 130 lines · RAJESWARAN THILLAIGOVINDAN
Apparao,
Thanks a lot for your suggestions. We lean towards using a dictionary to map Redfish ResourceType to D-Bus objects path and vice versa and then using D-Bus match to generate life cycle events. This way, the changes are limited to bmcweb. The resource type and the origin resource URI will be included in the event log. This requires change in the format of event log file
/var/log/redfish. I have commented the same in the server sent event patch that you have uploaded. Kindly see if you can leave the parsing of file to the OEMs. That way, the existing infrastructure can be used by the OEMs to support other filtering mechanisms as defined in the specification.Thanks,
RajesQuoted 120 lines · Apparao
Hi Rajeswaran,
Thanks for your mail. At the moment, I don't have plans to support the "ResourceTypes", "OriginResources" based filtering. Basically Intel uses file systems based redfish event
logs( journalctl -> rsync-> filesystem)and doesn't use D-Bus mechanism like IBM uses. So I am not much familiar with D-Bus logging but some of the suggestions:1) While logging redfish events over D-Bus itself, it can provide details on ResourceTypes and OriginResource URI/Path.
This is ideal and most efficient way. Since we walked a walked long distance from start, Its hard to modify all the services to uses these 2 new input parameters while logging events( Requires change in almost all repo's)
2) For resourcesTypes: Can have mapping dictionary against all MessageId's. For OriginResources: I believe, event log over D-Bus is already holding the Path. If so, last 3/4 nodes of uri can be taken and mapped against the resources and that can be used in Event filtering. We did used same mechanism in case of telemetry while mapping MetricReportDefinitions to URI.
Hope this helps.
Thanks,
-Appu
Quoted 101 lines · RAJESWARAN THILLAIGOVINDAN
Apparao,
I see that you have uploaded a patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for supporting server sent events. This patch supports event filtering based on registry prefix and/or messageId.
I would like to know if you have any plan to support event filtering based on resource type. If so, I would like to work together for a better solution. Earlier, I have proposed a solution based on D-Bus match using a dictionary. With that approach, the major challenge is to map Redfish resource and properties to D-Bus object and properties respectively. If D-Bus applications are modified to include resource type and origin of condition in the event, then there is no need for a map. But,that brings Redfish terminology to the application. Also, this will become an overhead if an OEM is not interested in Redfish event service.
Thanks,
RajesQuoted 91 lines · Apparao
Hi Ratan,
Comments in-line
Quoted 16 lines · Ratan Gupta
Hi Apparao,
Quoted 5 lines · Apparao
Hi,
I am sorry for late response as this mail is buried under and got struck back of my mind.
As i did mentioned in EventService design document, EventLog Monitor service is not common across the organizations( Example: Intel uses the redfish event logs file and inotify mechanism for monitoring the event logs. Where as IBM uses d-bus event log mechanism and they can relay on match rules). That said challenges with ResourceType mapping will be different in both monitoring mechanisms. This is good point. Initially when i started EventService design, i thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) but not sure that may become difficult when we expand supported ResourceTypes.
If I am getting correctly, Here is the flow which Intel uses.
1. Individual repo have to push the logs using
sd_journal_sendwhich will write to the file(/var/log/redfish) by using rsyslog daemonQuoted 8 lines
sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <
LOG_LEVEL>,"REDFISH_MESSAGE_ID=%s", "ResourceEvent.1.0.ResourceCreated",NULL); * How you would populate the "OriginOfCondition" during sending of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)Currently in logServices( logEntry), we are not reporting the "OriginOfCondition" property as per schema. I will check with Jason( Who wrote the logService) and get back on this.
BTW can you give me how this information is fetched in IBM way of LogService implementation( D-Bus)? If you already ratified any such, i think we can leverage. If not, We work together for solution.
Quoted 4 lines
* Any plan to add resource path in this journal message which tells that this is the path for which resource created event generated.
Same as above.
Quoted 3 lines
* Would the path be "Redfish Path/ D-bus Path"?
As per Redfish specification, This should be "@odata.id" which means it should be of resource uri and so we can't use d-bus path here for OriginOfConditions.
Quoted 7 lines
* Where the mapping would be done between D-busPath/Redfish Resource path?
Cons: Every application have to make the
change(use sd_journal_send). My thought is backend application should not be aware of the redfish terminlogy.Having separate process only for this mapping may not be good( No different from maintaining that map inside bmcweb as there won't be any other consumers). Ideal way is, that should be mapped while logging logEntry's itself. But we are not doing it currently which need to be re-looked. Give me some time, I will think and check with other folks and get back.
Quoted 6 lines
*2.* Some application(bmcweb) would do the Inotify on the path(/var/log/redfish) and send the event once there is any activity on this file.
Quoted 1 line
I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
per Intel design)
Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
Initially i thought "ResourceType" based event filtering can be done using minimal mapping( Using MessageID and args). But that will work for minimal set. If the supported ResourceTypes grows, we will have bigger challenges which i can sense it now. Anyway, Supported Resources are completely implementation specific. If this value is empty means, by default all event logs will be sent to subscribers. This is what we can start with before supported ResourceTypes list grows.
Quoted 31 lines
Quoted 27 lines
As per my reading from below query, You are looking at d-bus match rules and ResourceTypes mapping which is more specific to d-bus event logging(IBM way of implementing event logging). reading it from journal logs will give more information but that will impact the performance to large extent. This might be one of the reason why we (Intel) uses Redfish message ID while logging redfish events logs to journal(You can refer design document for same at https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). In opinion, in your d-bus if you are using some kind of
filter(Example REDFISH_MESSAGE_ID)while logging in journal logs for all events and figure out the way to monitor the journal logs without impacting the performance, that should be ok as long as match filters are satisfied for Redfish EventService subscriptions and supported Types(Again differs with implementation).Thanks,
-Appu
Quoted 19 lines · RAJESWARAN THILLAIGOVINDAN
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
RajesQuoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
RajesThanks
Ratan
Hi Appu,
Can you get me an example say if I want an redfish event to be generated once any local user gets added on the BMC.
What would be the steps to be done with the current design and where(which repo)?
Quoted 5 lines · Apparao
Rajes,
The dictionary to map Redfish resourceType and D-Bus object path( I believe URI intern) is good. It will be great if you share design document, if its done.
Quoted 137 lines
At the moment, redfish event logs file(/var/log/redfish) doesn't have ResourceTypes and OriginResource fields. The Existing redfish event log implementation(log service) also doesn't have support for that. You can propose design change for same along with how it is used in event log service. Same thing, can be adopted to EventService, once its agreed by OEM's( I am thinking, it should go under new OEM specific compiler flag. But that we can ratify later).
Thanks,
-Appu
Quoted 130 lines · RAJESWARAN THILLAIGOVINDAN
Apparao,
Thanks a lot for your suggestions. We lean towards using a dictionary to map Redfish ResourceType to D-Bus objects path and vice versa and then using D-Bus match to generate life cycle events. This way, the changes are limited to bmcweb. The resource type and the origin resource URI will be included in the event log. This requires change in the format of event log file
/var/log/redfish. I have commented the same in the server sent event patch that you have uploaded. Kindly see if you can leave the parsing of file to the OEMs. That way, the existing infrastructure can be used by the OEMs to support other filtering mechanisms as defined in the specification.Thanks,
RajesQuoted 120 lines · Apparao
Hi Rajeswaran,
Thanks for your mail. At the moment, I don't have plans to support the "ResourceTypes", "OriginResources" based filtering. Basically Intel uses file systems based redfish event
logs( journalctl -> rsync-> filesystem)and doesn't use D-Bus mechanism like IBM uses. So I am not much familiar with D-Bus logging but some of the suggestions:1) While logging redfish events over D-Bus itself, it can provide details on ResourceTypes and OriginResource URI/Path.
This is ideal and most efficient way. Since we walked a walked long distance from start, Its hard to modify all the services to uses these 2 new input parameters while logging events( Requires change in almost all repo's)
2) For resourcesTypes: Can have mapping dictionary against all MessageId's. For OriginResources: I believe, event log over D-Bus is already holding the Path. If so, last 3/4 nodes of uri can be taken and mapped against the resources and that can be used in Event filtering. We did used same mechanism in case of telemetry while mapping MetricReportDefinitions to URI.
Hope this helps.
Thanks,
-Appu
Quoted 101 lines · RAJESWARAN THILLAIGOVINDAN
Apparao,
I see that you have uploaded a patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for supporting server sent events. This patch supports event filtering based on registry prefix and/or messageId.
I would like to know if you have any plan to support event filtering based on resource type. If so, I would like to work together for a better solution. Earlier, I have proposed a solution based on D-Bus match using a dictionary. With that approach, the major challenge is to map Redfish resource and properties to D-Bus object and properties respectively. If D-Bus applications are modified to include resource type and origin of condition in the event, then there is no need for a map. But,that brings Redfish terminology to the application. Also, this will become an overhead if an OEM is not interested in Redfish event service.
Thanks,
RajesQuoted 91 lines · Apparao
Hi Ratan,
Comments in-line
Quoted 16 lines · Ratan Gupta
Hi Apparao,
Quoted 5 lines · Apparao
Hi,
I am sorry for late response as this mail is buried under and got struck back of my mind.
As i did mentioned in EventService design document, EventLog Monitor service is not common across the organizations( Example: Intel uses the redfish event logs file and inotify mechanism for monitoring the event logs. Where as IBM uses d-bus event log mechanism and they can relay on match rules). That said challenges with ResourceType mapping will be different in both monitoring mechanisms. This is good point. Initially when i started EventService design, i thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) but not sure that may become difficult when we expand supported ResourceTypes.
If I am getting correctly, Here is the flow which Intel uses.
1. Individual repo have to push the logs using
sd_journal_sendwhich will write to the file(/var/log/redfish) by using rsyslog daemonQuoted 8 lines
sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <
LOG_LEVEL>,"REDFISH_MESSAGE_ID=%s", "ResourceEvent.1.0.ResourceCreated",NULL); * How you would populate the "OriginOfCondition" during sending of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)Currently in logServices( logEntry), we are not reporting the "OriginOfCondition" property as per schema. I will check with Jason( Who wrote the logService) and get back on this.
BTW can you give me how this information is fetched in IBM way of LogService implementation( D-Bus)? If you already ratified any such, i think we can leverage. If not, We work together for solution.
Quoted 4 lines
* Any plan to add resource path in this journal message which tells that this is the path for which resource created event generated.
Same as above.
Quoted 3 lines
* Would the path be "Redfish Path/ D-bus Path"?
As per Redfish specification, This should be "@odata.id" which means it should be of resource uri and so we can't use d-bus path here for OriginOfConditions.
Quoted 7 lines
* Where the mapping would be done between D-busPath/Redfish Resource path?
Cons: Every application have to make the
change(use sd_journal_send). My thought is backend application should not be aware of the redfish terminlogy.Having separate process only for this mapping may not be good( No different from maintaining that map inside bmcweb as there won't be any other consumers). Ideal way is, that should be mapped while logging logEntry's itself. But we are not doing it currently which need to be re-looked. Give me some time, I will think and check with other folks and get back.
Quoted 6 lines
*2.* Some application(bmcweb) would do the Inotify on the path(/var/log/redfish) and send the event once there is any activity on this file.
Quoted 1 line
I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
per Intel design)
Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
Initially i thought "ResourceType" based event filtering can be done using minimal mapping( Using MessageID and args). But that will work for minimal set. If the supported ResourceTypes grows, we will have bigger challenges which i can sense it now. Anyway, Supported Resources are completely implementation specific. If this value is empty means, by default all event logs will be sent to subscribers. This is what we can start with before supported ResourceTypes list grows.
Quoted 31 lines
Quoted 27 lines
As per my reading from below query, You are looking at d-bus match rules and ResourceTypes mapping which is more specific to d-bus event logging(IBM way of implementing event logging). reading it from journal logs will give more information but that will impact the performance to large extent. This might be one of the reason why we (Intel) uses Redfish message ID while logging redfish events logs to journal(You can refer design document for same at https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). In opinion, in your d-bus if you are using some kind of
filter(Example REDFISH_MESSAGE_ID)while logging in journal logs for all events and figure out the way to monitor the journal logs without impacting the performance, that should be ok as long as match filters are satisfied for Redfish EventService subscriptions and supported Types(Again differs with implementation).Thanks,
-Appu
Quoted 19 lines · RAJESWARAN THILLAIGOVINDAN
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
RajesQuoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
RajesThanks
Ratan
Ratan
Re: Redfish event log for new local user addition
[Subject changed to reflect actual context]
Hi Ratan,
With current OpenBMC, I don't any event log getting added for new local user creation. To implement that below are two place we need to add support:
1) Define an new message retry ID for user addition(say UserAdded) in bmcweb.
2) Add a event log during new user addition inside phospor-user-manager(createUser function).
https://github.com/openbmc/phosphor-user-manager/blob/master/user_mgr.cpp#L336
While you are adding event log for new local user creation, Please consider other case too( Like Delete, Update, Rename etc..)
Thanks,
-Appu
Quoted 152 lines · Ratan Gupta
Hi Appu,
Can you get me an example say if I want an redfish event to be generated once any local user gets added on the BMC.
What would be the steps to be done with the current design and where(which repo)?
Quoted 5 lines · Apparao
Rajes,
The dictionary to map Redfish resourceType and D-Bus object path( I believe URI intern) is good. It will be great if you share design document, if its done.
Quoted 137 lines
At the moment, redfish event logs file(/var/log/redfish) doesn't have ResourceTypes and OriginResource fields. The Existing redfish event log implementation(log service) also doesn't have support for that. You can propose design change for same along with how it is used in event log service. Same thing, can be adopted to EventService, once its agreed by OEM's( I am thinking, it should go under new OEM specific compiler flag. But that we can ratify later).
Thanks,
-Appu
Quoted 130 lines · RAJESWARAN THILLAIGOVINDAN
Apparao,
Thanks a lot for your suggestions. We lean towards using a dictionary to map Redfish ResourceType to D-Bus objects path and vice versa and then using D-Bus match to generate life cycle events. This way, the changes are limited to bmcweb. The resource type and the origin resource URI will be included in the event log. This requires change in the format of event log file
/var/log/redfish. I have commented the same in the server sent event patch that you have uploaded. Kindly see if you can leave the parsing of file to the OEMs. That way, the existing infrastructure can be used by the OEMs to support other filtering mechanisms as defined in the specification.Thanks,
RajesQuoted 120 lines · Apparao
Hi Rajeswaran,
Thanks for your mail. At the moment, I don't have plans to support the "ResourceTypes", "OriginResources" based filtering. Basically Intel uses file systems based redfish event
logs( journalctl -> rsync-> filesystem)and doesn't use D-Bus mechanism like IBM uses. So I am not much familiar with D-Bus logging but some of the suggestions:1) While logging redfish events over D-Bus itself, it can provide details on ResourceTypes and OriginResource URI/Path.
This is ideal and most efficient way. Since we walked a walked long distance from start, Its hard to modify all the services to uses these 2 new input parameters while logging events( Requires change in almost all repo's)
2) For resourcesTypes: Can have mapping dictionary against all MessageId's. For OriginResources: I believe, event log over D-Bus is already holding the Path. If so, last 3/4 nodes of uri can be taken and mapped against the resources and that can be used in Event filtering. We did used same mechanism in case of telemetry while mapping MetricReportDefinitions to URI.
Hope this helps.
Thanks,
-Appu
Quoted 101 lines · RAJESWARAN THILLAIGOVINDAN
Apparao,
I see that you have uploaded a patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for supporting server sent events. This patch supports event filtering based on registry prefix and/or messageId.
I would like to know if you have any plan to support event filtering based on resource type. If so, I would like to work together for a better solution. Earlier, I have proposed a solution based on D-Bus match using a dictionary. With that approach, the major challenge is to map Redfish resource and properties to D-Bus object and properties respectively. If D-Bus applications are modified to include resource type and origin of condition in the event, then there is no need for a map. But,that brings Redfish terminology to the application. Also, this will become an overhead if an OEM is not interested in Redfish event service.
Thanks,
RajesQuoted 91 lines · Apparao
Hi Ratan,
Comments in-line
Quoted 16 lines · Ratan Gupta
Hi Apparao,
Quoted 5 lines · Apparao
Hi,
I am sorry for late response as this mail is buried under and got struck back of my mind.
As i did mentioned in EventService design document, EventLog Monitor service is not common across the organizations( Example: Intel uses the redfish event logs file and inotify mechanism for monitoring the event logs. Where as IBM uses d-bus event log mechanism and they can relay on match rules). That said challenges with ResourceType mapping will be different in both monitoring mechanisms. This is good point. Initially when i started EventService design, i thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) but not sure that may become difficult when we expand supported ResourceTypes.
If I am getting correctly, Here is the flow which Intel uses.
1. Individual repo have to push the logs using
sd_journal_sendwhich will write to the file(/var/log/redfish) by using rsyslog daemonQuoted 8 lines
sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <
LOG_LEVEL>,"REDFISH_MESSAGE_ID=%s", "ResourceEvent.1.0.ResourceCreated",NULL); * How you would populate the "OriginOfCondition" during sending of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)Currently in logServices( logEntry), we are not reporting the "OriginOfCondition" property as per schema. I will check with Jason( Who wrote the logService) and get back on this.
BTW can you give me how this information is fetched in IBM way of LogService implementation( D-Bus)? If you already ratified any such, i think we can leverage. If not, We work together for solution.
Quoted 4 lines
* Any plan to add resource path in this journal message which tells that this is the path for which resource created event generated.
Same as above.
Quoted 3 lines
* Would the path be "Redfish Path/ D-bus Path"?
As per Redfish specification, This should be "@odata.id" which means it should be of resource uri and so we can't use d-bus path here for OriginOfConditions.
Quoted 7 lines
* Where the mapping would be done between
D-busPath/Redfish Resource path?Cons: Every application have to make the
change(use sd_journal_send). My thought is backend application should not be aware of the redfish terminlogy.Having separate process only for this mapping may not be good( No different from maintaining that map inside bmcweb as there won't be any other consumers). Ideal way is, that should be mapped while logging logEntry's itself. But we are not doing it currently which need to be re-looked. Give me some time, I will think and check with other folks and get back.
Quoted 6 lines
*2.* Some application(bmcweb) would do the Inotify on the path(/var/log/redfish) and send the event once there is any activity on this file.
Quoted 1 line
I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
per Intel design)
Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
Initially i thought "ResourceType" based event filtering can be done using minimal mapping( Using MessageID and args). But that will work for minimal set. If the supported ResourceTypes grows, we will have bigger challenges which i can sense it now. Anyway, Supported Resources are completely implementation specific. If this value is empty means, by default all event logs will be sent to subscribers. This is what we can start with before supported ResourceTypes list grows.
Quoted 31 lines
Quoted 27 lines
As per my reading from below query, You are looking at d-bus match rules and ResourceTypes mapping which is more specific to d-bus event logging(IBM way of implementing event logging). reading it from journal logs will give more information but that will impact the performance to large extent. This might be one of the reason why we (Intel) uses Redfish message ID while logging redfish events logs to journal(You can refer design document for same at https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). In opinion, in your d-bus if you are using some kind of
filter(Example REDFISH_MESSAGE_ID)while logging in journal logs for all events and figure out the way to monitor the journal logs without impacting the performance, that should be ok as long as match filters are satisfied for Redfish EventService subscriptions and supported Types(Again differs with implementation).Thanks,
-Appu
Quoted 19 lines · RAJESWARAN THILLAIGOVINDAN
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
RajesQuoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
RajesThanks
RatanRatan
Re: Redfish event log for new local user addition
Hi Appu,
Thanks for responding.
As the Impact would be on all the openbmc repo where we need to add the event log for any properties change or interface added.
I feel it is kind of overhead and not realistic as each and every repo need to know about redfish terminology eg : Redfish Path, Redfish property, Redfish Registry Prefix etc.
I feel it is a big work.
Ratan
Quoted 176 lines · Apparao
[Subject changed to reflect actual context]
Hi Ratan,
With current OpenBMC, I don't any event log getting added for new local user creation. To implement that below are two place we need to add support:
1) Define an new message retry ID for user addition(say UserAdded) in bmcweb.
2) Add a event log during new user addition inside phospor-user-manager(createUser function).
https://github.com/openbmc/phosphor-user-manager/blob/master/user_mgr.cpp#L336
While you are adding event log for new local user creation, Please consider other case
too( Like Delete, Update, Rename etc..)Thanks,
-Appu
Quoted 152 lines · Ratan Gupta
Hi Appu,
Can you get me an example say if I want an redfish event to be generated once any local user gets added on the BMC.
What would be the steps to be done with the current design and where(which repo)?
Quoted 5 lines · Apparao
Rajes,
The dictionary to map Redfish resourceType and D-Bus object path( I believe URI intern) is good. It will be great if you share design document, if its done.
Quoted 137 lines
At the moment, redfish event logs file(/var/log/redfish) doesn't have ResourceTypes and OriginResource fields. The Existing redfish event log implementation(log service) also doesn't have support for that. You can propose design change for same along with how it is used in event log service. Same thing, can be adopted to EventService, once its agreed by OEM's( I am thinking, it should go under new OEM specific compiler flag. But that we can ratify later).
Thanks,
-Appu
Quoted 130 lines · RAJESWARAN THILLAIGOVINDAN
Apparao,
Thanks a lot for your suggestions. We lean towards using a dictionary to map Redfish ResourceType to D-Bus objects path and vice versa and then using D-Bus match to generate life cycle events. This way, the changes are limited to bmcweb. The resource type and the origin resource URI will be included in the event log. This requires change in the format of event log file
/var/log/redfish. I have commented the same in the server sent event patch that you have uploaded. Kindly see if you can leave the parsing of file to the OEMs. That way, the existing infrastructure can be used by the OEMs to support other filtering mechanisms as defined in the specification.Thanks,
RajesQuoted 120 lines · Apparao
Hi Rajeswaran,
Thanks for your mail. At the moment, I don't have plans to support the "ResourceTypes", "OriginResources" based filtering. Basically Intel uses file systems based redfish event
logs( journalctl -> rsync-> filesystem)and doesn't use D-Bus mechanism like IBM uses. So I am not much familiar with D-Bus logging but some of the suggestions:1) While logging redfish events over D-Bus itself, it can provide details on ResourceTypes and OriginResource URI/Path.
This is ideal and most efficient way. Since we walked a walked long distance from start, Its hard to modify all the services to uses these 2 new input parameters while logging events( Requires change in almost all repo's)
2) For resourcesTypes: Can have mapping dictionary against all MessageId's. For OriginResources: I believe, event log over D-Bus is already holding the Path. If so, last 3/4 nodes of uri can be taken and mapped against the resources and that can be used in Event filtering. We did used same mechanism in case of telemetry while mapping MetricReportDefinitions to URI.
Hope this helps.
Thanks,
-Appu
Quoted 101 lines · RAJESWARAN THILLAIGOVINDAN
Apparao,
I see that you have uploaded a patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for supporting server sent events. This patch supports event filtering based on registry prefix and/or messageId.
I would like to know if you have any plan to support event filtering based on resource type. If so, I would like to work together for a better solution. Earlier, I have proposed a solution based on D-Bus match using a dictionary. With that approach, the major challenge is to map Redfish resource and properties to D-Bus object and properties respectively. If D-Bus applications are modified to include resource type and origin of condition in the event, then there is no need for a map. But,that brings Redfish terminology to the application. Also, this will become an overhead if an OEM is not interested in Redfish event service.
Thanks,
RajesQuoted 91 lines · Apparao
Hi Ratan,
Comments in-line
Quoted 16 lines · Ratan Gupta
Hi Apparao,
Quoted 5 lines · Apparao
Hi,
I am sorry for late response as this mail is buried under and got struck back of my mind.
As i did mentioned in EventService design document, EventLog Monitor service is not common across the organizations( Example: Intel uses the redfish event logs file and inotify mechanism for monitoring the event logs. Where as IBM uses d-bus event log mechanism and they can relay on match rules). That said challenges with ResourceType mapping will be different in both monitoring mechanisms. This is good point. Initially when i started EventService design, i thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) but not sure that may become difficult when we expand supported ResourceTypes.
If I am getting correctly, Here is the flow which Intel uses.
1. Individual repo have to push the logs using
sd_journal_sendwhich will write to the file(/var/log/redfish) by using rsyslog daemonQuoted 8 lines
sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <
LOG_LEVEL>,"REDFISH_MESSAGE_ID=%s", "ResourceEvent.1.0.ResourceCreated",NULL); * How you would populate the "OriginOfCondition" during sending of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)Currently in logServices( logEntry), we are not reporting the "OriginOfCondition" property as per schema. I will check with Jason( Who wrote the logService) and get back on this.
BTW can you give me how this information is fetched in IBM way of LogService implementation( D-Bus)? If you already ratified any such, i think we can leverage. If not, We work together for solution.
Quoted 4 lines
* Any plan to add resource path in this journal message which tells that this is the path for which resource created event generated.
Same as above.
Quoted 3 lines
* Would the path be "Redfish Path/ D-bus Path"?
As per Redfish specification, This should be "@odata.id" which means it should be of resource uri and so we can't use d-bus path here for OriginOfConditions.
Quoted 7 lines
* Where the mapping would be done between
D-busPath/Redfish Resource path?Cons: Every application have to make the
change(use sd_journal_send). My thought is backend application should not be aware of the redfish terminlogy.Having separate process only for this mapping may not be good( No different from maintaining that map inside bmcweb as there won't be any other consumers). Ideal way is, that should be mapped while logging logEntry's itself. But we are not doing it currently which need to be re-looked. Give me some time, I will think and check with other folks and get back.
Quoted 6 lines
*2.* Some application(bmcweb) would do the Inotify on the path(/var/log/redfish) and send the event once there is any activity on this file.
Quoted 1 line
I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
per Intel design)
Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
Initially i thought "ResourceType" based event filtering can be done using minimal mapping( Using MessageID and args). But that will work for minimal set. If the supported ResourceTypes grows, we will have bigger challenges which i can sense it now. Anyway, Supported Resources are completely implementation specific. If this value is empty means, by default all event logs will be sent to subscribers. This is what we can start with before supported ResourceTypes list grows.
Quoted 31 lines
Quoted 27 lines
As per my reading from below query, You are looking at d-bus match rules and ResourceTypes mapping which is more specific to d-bus event logging(IBM way of implementing event logging). reading it from journal logs will give more information but that will impact the performance to large extent. This might be one of the reason why we (Intel) uses Redfish message ID while logging redfish events logs to journal(You can refer design document for same at https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). In opinion, in your d-bus if you are using some kind of
filter(Example REDFISH_MESSAGE_ID)while logging in journal logs for all events and figure out the way to monitor the journal logs without impacting the performance, that should be ok as long as match filters are satisfied for Redfish EventService subscriptions and supported Types(Again differs with implementation).Thanks,
-Appu
Quoted 19 lines · RAJESWARAN THILLAIGOVINDAN
ApparaRao.
As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in.
Thanks,
RajesQuoted 10 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either thephosphor::logging::entryorsd_journal_send()command.After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ?
Your views are highly appreciated.
Thanks,
RajesThanks
RatanRatan
Quoted 6 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging.
Would anyone else be willing to opine on whether or not they think journal based event schemes are what we want going forward for OpenBMC?
My feeling is that they are not - as an alternative IPC mechanism don't we end up re-implementing things that DBus already does? Doesn't it require us to raise the same event twice everywhere (Once with DBus, and once in the journal)? What does journal based eventing do that DBus signals don't do?
Please poke holes.
thx - brad
Quoted 8 lines · Brad Bishop
Quoted 6 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging.
Quoted 8 lines
Would anyone else be willing to opine on whether or not they think journal based event schemes are what we want going forward for OpenBMC?
My feeling is that they are not - as an alternative IPC mechanism don't we end up re-implementing things that DBus already does? Doesn't it require us to raise the same event twice everywhere (Once with DBus, and once in the journal)? What does journal based eventing do that DBus signals don't do?
We don't host log events on DBus at all, so there is no duplicate. The journal gives built in persistence and rotating of logs for large number of events. I know when this came up the last time one of the big issues was supporting thousands of logs wouldn't work well on DBus.
Quoted 5 lines
Please poke holes.
thx - brad
Hi James/Apparao/Brad,
I am inclining towards having a separate application for Redfish Logs(like: phosphor-sel-logger), This application does the following.
1) Have the mapping info from Redfish resources to Dbus Resources
* This is needed as webserver have to provide the event filtering through Resource Type
* eg : Redfish Client may ask as the client is interested in "Account" Resource type i.e all the user account related updates should be given to redfish client. which suggest that there should be mapping from Redfish Resource to Dbus Resource
2) Have the reverse mapping from Dbus Resources to Redfish Resources
* This is needed to send the Redfish event if there is any changes in the corresponding D-bus resources. eg BMC state change/network change etc.
3) This application monitors the D-bus event and Log the event in the journal like below eg:
sd_journal_send("MESSAGE=%s", "Account Modified",
"PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s", "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>", "REDFISH_RESOURCE_TYPE=ComputerSystem" "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
4) rsyslogd will put all these logs from journal into "/var/log/redfish" file.
5) bmcweb/webserver would inotify this file location and on notification it will send redfish event
6) Event filtering would be done at the bmcweb/webserver side.
We already have the infrastructure for seq no 4 and seq no 5 and we wanted to leverage this infrastructure.
Please let me know if there is any concern with this approach.
Ratan
Quoted 26 lines · James Feist
Quoted 8 lines · Brad Bishop
Quoted 6 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging.Quoted 8 lines
Would anyone else be willing to opine on whether or not they think journal based event schemes are what we want going forward for OpenBMC?
My feeling is that they are not - as an alternative IPC mechanism don't we end up re-implementing things that DBus already does? Doesn't it require us to raise the same event twice everywhere (Once with DBus, and once in the journal)? What does journal based eventing do that DBus signals don't do?
We don't host log events on DBus at all, so there is no duplicate. The journal gives built in persistence and rotating of logs for large number of events. I know when this came up the last time one of the big issues was supporting thousands of logs wouldn't work well on DBus.
Quoted 5 lines
Please poke holes.
thx - brad
Quoted 15 lines · Ratan Gupta
Hi James/Apparao/Brad,
I am inclining towards having a separate application for Redfish Logs(like: phosphor-sel-logger), This application does the following.
1) Have the mapping info from Redfish resources to Dbus Resources
* This is needed as webserver have to provide the event filtering
through Resource Type
* eg : Redfish Client may ask as the client is interested in
"Account" Resource type
i.e all the user account related updates should be given to
redfish client.
which suggest that there should be mapping from Redfish
Resource to Dbus Resource
Quoted 7 lines
2) Have the reverse mapping from Dbus Resources to Redfish Resources
* This is needed to send the Redfish event if there is any changes
in the
corresponding D-bus resources. eg BMC state change/network
change etc.3) This application monitors the D-bus event and Log the event in the journal like below
Why do we need a listener on d-bus if the application creating the event already knows all the information? Why can't it log straight to the journal?
Quoted 7 lines
eg:
sd_journal_send("MESSAGE=%s", "Account Modified", "PRIORITY=%i",LOG_INFO, "REDFISH_MESSAGE_ID=%s",
"REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
"REDFISH_RESOURCE_TYPE=ComputerSystem" "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
If we're going to go down the path of re-implementing logging, I think the goal should be to stop logging things in the Journal that are Redfish specific, and instead log them in some generic format that phosphor logging controls the map. Right now we are bifurcated because the dbus-event-logs, SEL, PEL, and Redfish are all using different methods of logging, that play to their own set of rules. Most repos use phosphor-logging, so instead of creating yet another daemon, if we added support to create a 'System Level' or 'External User' log that has predefined dictionaries of required and optional keys, something like phosphor-dbus-interfaces, we might be able to drop some of these transport specific logs, and have the transports based on the same database (the journal). Then each transport could filter these based on journal entry type, and transform them into the correct log for that transport. I think adding more Redfish specifics into the logs hinders those who do not want Redfish in their systems.
Quoted 7 lines
4) rsyslogd will put all these logs from journal into "
/var/log/redfish" file.5) bmcweb/webserver would inotify this file location and on notification it will send redfish event
6) Event filtering would be done at the bmcweb/webserver side.
I think this will probably be needed if we want one source of truth for logs.
Quoted 34 lines
We already have the infrastructure for seq no 4 and seq no 5 and we wanted to leverage this infrastructure.
Please let me know if there is any concern with this approach.
Ratan
Quoted 26 lines · James Feist
Quoted 8 lines · Brad Bishop
Quoted 6 lines · RAJESWARAN THILLAIGOVINDAN
Hi,
I am going through the bmcweb code for implementing Redfish EventService based on the design document
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging.Quoted 8 lines
Would anyone else be willing to opine on whether or not they think journal based event schemes are what we want going forward for OpenBMC?
My feeling is that they are not - as an alternative IPC mechanism don't we end up re-implementing things that DBus already does? Doesn't it require us to raise the same event twice everywhere (Once with DBus, and once in the journal)? What does journal based eventing do that DBus signals don't do?
We don't host log events on DBus at all, so there is no duplicate. The journal gives built in persistence and rotating of logs for large number of events. I know when this came up the last time one of the big issues was supporting thousands of logs wouldn't work well on DBus.
Quoted 5 lines
Please poke holes.
thx - brad
Quoted 9 lines · James Feist
Quoted 7 lines · Ratan Gupta
eg:
sd_journal_send("MESSAGE=%s", "Account Modified", "PRIORITY=%i",LOG_INFO, "REDFISH_MESSAGE_ID=%s",
"REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
"REDFISH_RESOURCE_TYPE=ComputerSystem" "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
Quoted 6 lines
If we're going to go down the path of re-implementing logging, I think the goal should be to stop logging things in the Journal that are Redfish specific, and instead log them in some generic format that phosphor logging controls the map. Right now we are bifurcated because the dbus-event-logs, SEL, PEL, and Redfish are all using different methods of logging, that play to their own set of rules.
Absolutely agree with you here. There is zero reason that applications should start logging specially formed messages with REDFISH_* meta-data. We shouldn't have any applications explicitly know about Redfish except the Redfish providers themselves. This is no different from IPMI, PLDM, or any other external interface.
The kind of information presented here as being interesting to expose via Redfish is equally as interesting to me to be able to add to one of our 'FFDC dumps', which could be used for security / forensic work.
Quoted 10 lines
Most repos use
phosphor-logging, so instead of creating yet another daemon, if we added support to create a 'System Level' or 'External User' log that has predefined dictionaries of required and optional keys, something like phosphor-dbus-interfaces, we might be able to drop some of these transport specific logs, and have the transports based on the same database (the journal). Then each transport could filter these based on journal entry type, and transform them into the correct log for that transport. I think adding more Redfish specifics into the logs hinders those who do not want Redfish in their systems.
Can't we do this already today by defining a simple errors/metadata file in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? This will create a record on dbus in phosphor-logging.
Signature · 1 line
Patrick Williams
Quoted 38 lines · Patrick Williams
Quoted 9 lines · James Feist
Quoted 7 lines · Ratan Gupta
eg:
sd_journal_send("MESSAGE=%s", "Account Modified", "PRIORITY=%i",LOG_INFO, "REDFISH_MESSAGE_ID=%s",
"REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
"REDFISH_RESOURCE_TYPE=ComputerSystem" "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);Quoted 6 lines
If we're going to go down the path of re-implementing logging, I think the goal should be to stop logging things in the Journal that are Redfish specific, and instead log them in some generic format that phosphor logging controls the map. Right now we are bifurcated because the dbus-event-logs, SEL, PEL, and Redfish are all using different methods of logging, that play to their own set of rules.
Absolutely agree with you here. There is zero reason that applications should start logging specially formed messages with REDFISH_* meta-data. We shouldn't have any applications explicitly know about Redfish except the Redfish providers themselves. This is no different from IPMI, PLDM, or any other external interface.
The kind of information presented here as being interesting to expose via Redfish is equally as interesting to me to be able to add to one of our 'FFDC dumps', which could be used for security / forensic work.
Quoted 10 lines
Most repos use
phosphor-logging, so instead of creating yet another daemon, if we added support to create a 'System Level' or 'External User' log that has predefined dictionaries of required and optional keys, something like phosphor-dbus-interfaces, we might be able to drop some of these transport specific logs, and have the transports based on the same database (the journal). Then each transport could filter these based on journal entry type, and transform them into the correct log for that transport. I think adding more Redfish specifics into the logs hinders those who do not want Redfish in their systems.
Quoted 4 lines
Can't we do this already today by defining a simple errors/metadata file in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? This will create a record on dbus in phosphor-logging.
I think the original concern was with supporting on the order of 10,000 log entries, having this on d-bus seemed impractical. Also the free log rotation the journal provides is useful. Now modifying the logging::report<...> to conditionally log to the journal seems realistic.
Hi James,Pattrick.
Thanks for giving the feedback
Quoted 39 lines · James Feist
Quoted 38 lines · Patrick Williams
Quoted 9 lines · James Feist
Quoted 7 lines · Ratan Gupta
eg:
sd_journal_send("MESSAGE=%s", "Account Modified", "PRIORITY=%i",LOG_INFO, "REDFISH_MESSAGE_ID=%s", "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>", "REDFISH_RESOURCE_TYPE=ComputerSystem" "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);Quoted 6 lines
If we're going to go down the path of re-implementing logging, I think the goal should be to stop logging things in the Journal that are Redfish specific, and instead log them in some generic format that phosphor logging controls the map. Right now we are bifurcated because the dbus-event-logs, SEL, PEL, and Redfish are all using different methods of logging, that play to their own set of rules.
Absolutely agree with you here. There is zero reason that applications should start logging specially formed messages with REDFISH_* meta-data. We shouldn't have any applications explicitly know about Redfish except the Redfish providers themselves. This is no different from IPMI, PLDM, or any other external interface.
The kind of information presented here as being interesting to expose via Redfish is equally as interesting to me to be able to add to one of our 'FFDC dumps', which could be used for security / forensic work.
Quoted 10 lines
Most repos use
phosphor-logging, so instead of creating yet another daemon, if we added support to create a 'System Level' or 'External User' log that has predefined dictionaries of required and optional keys, something like phosphor-dbus-interfaces, we might be able to drop some of these transport specific logs, and have the transports based on the same database (the journal). Then each transport could filter these based on journal entry type, and transform them into the correct log for that transport. I think adding more Redfish specifics into the logs hinders those who do not want Redfish in their systems.
Quoted 5 lines
Quoted 4 lines
Can't we do this already today by defining a simple errors/metadata file in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? This will create a record on dbus in phosphor-logging.
I think the original concern was with supporting on the order of 10,000 log entries, having this on d-bus seemed impractical. Also the free log rotation the journal provides is useful. Now modifying the logging::report<...> to conditionally log to the journal seems realistic.
My intention was not to re-implement the logging, my intention was to extend/use the existing design which we already have it below.
https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
I was trying not to bring the Redfish specific stuff in each individual repo, instead each transport can listen for Dbus events and write to the journal which goes to their app specific file.
Your point is valid that we would be polluting the journal if each and every transport start writing as per their needs.
***** As per the Redfish one of the requirement is we need the log for most of the Dbus Property update/interface added as they are mapped to some Redfish Resource and the bmcweb has to send the Resource updated/modified signal to the Redfish client. ******
As we are in agreement that we want to use the journal for persistence and log rotate feature.
We have two options:
1) Each transport interface listens for the Dbus signals and write it to their app specific file.
2) Each openbmc repo must use log::report for each D-bus property update/ interface added. To make this happen, we need the following
* common message format which can be used by the all other application (SEL,Redfish) etc.
* Transport application will read this common message from the journal and use it as per their needs.
Option no 2: It is ideal solution but that requires good amount of work as
either each and every D-bus repo should write to the journal for any property update/interface added. or
Should SDbusplus do this as each and every application uses the sdbusplus framework to manage the dbus objects
Please let me know your thoughts.
Ratan
Quoted 14 lines
My intention was not to re-implement the logging, my intention was to extend/use the existing design which we already have it below.
https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
I was trying not to bring the Redfish specific stuff in each individual repo, instead each transport can listen for Dbus events and write to the journal which goes to their app specific file.
Your point is valid that we would be polluting the journal if each and every transport start writing as per their needs.
***** As per the Redfish one of the requirement is we need the log for most of the Dbus Property update/interface added as they are mapped to some Redfish Resource and the bmcweb has to send the Resource updated/modified signal to the Redfish client. ******
Quoted 7 lines
As we are in agreement that we want to use the journal for persistence and log rotate feature.
We have two options:
1) Each transport interface listens for the Dbus signals and write
it to their app specific file.
2) Each openbmc repo must uselog::reportfor each D-bus property
update/ interface added.
I'm assuming you meant each event added right? I don't see any reason that the logs have anything to do with d-bus, except maybe if it seems worth it for phosphor-logging to catch all the reports and log them to the journal. Although it would probably be more efficient (and easier for phased adoption) to just make the log::report call write to the journal as well.
Quoted 3 lines
To make this happen, we need the following
* common message format which can be used by the all other
application (SEL,Redfish) etc.
Agree.
Quoted 6 lines
* Transport application will read this common message from the
journal and use it as per their needs.Option no 2: It is ideal solution but that requires good amount of work as
either each and every D-bus repo should write to the
journal for any property update/interface added.
Why every property/interface added? This would make the logs much to large to be useful. And many times the information needed is specific to the log type. For instance a event for inventory discovered might need the type of inventory, the serial number, the model, etc. To get all that information would be multiple interfaces added events combined together. It would make more sense in this aspect for the inventory daemon to take the parts out of say the Asset, and Chassis interfaces along with the path identifier, and create an entry in the log that is use-able. It would be much more difficult and error-prone for the Redfish or SEL daemon to try to piece these multiple events together to form something useful.
Quoted 8 lines
or
Should SDbusplus do this as each and every application
uses the sdbusplus framework to manage the dbus objectsPlease let me know your thoughts.
Ratan
Quoted 54 lines · Ratan Gupta
Hi James,Pattrick.
Thanks for giving the feedback
Quoted 39 lines · James Feist
Quoted 38 lines · Patrick Williams
Quoted 9 lines · James Feist
Quoted 7 lines · Ratan Gupta
eg:
sd_journal_send("MESSAGE=%s", "Account Modified", "PRIORITY=%i",LOG_INFO, "REDFISH_MESSAGE_ID=%s", "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>", "REDFISH_RESOURCE_TYPE=ComputerSystem" "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);Quoted 6 lines
If we're going to go down the path of re-implementing logging, I think the goal should be to stop logging things in the Journal that are Redfish specific, and instead log them in some generic format that phosphor logging controls the map. Right now we are bifurcated because the dbus-event-logs, SEL, PEL, and Redfish are all using different methods of logging, that play to their own set of rules.
Absolutely agree with you here. There is zero reason that applications should start logging specially formed messages with REDFISH_* meta-data. We shouldn't have any applications explicitly know about Redfish except the Redfish providers themselves. This is no different from IPMI, PLDM, or any other external interface.
The kind of information presented here as being interesting to expose via Redfish is equally as interesting to me to be able to add to one of our 'FFDC dumps', which could be used for security / forensic work.
Quoted 10 lines
Most repos use
phosphor-logging, so instead of creating yet another daemon, if we added support to create a 'System Level' or 'External User' log that has predefined dictionaries of required and optional keys, something like phosphor-dbus-interfaces, we might be able to drop some of these transport specific logs, and have the transports based on the same database (the journal). Then each transport could filter these based on journal entry type, and transform them into the correct log for that transport. I think adding more Redfish specifics into the logs hinders those who do not want Redfish in their systems.Quoted 5 lines
Quoted 4 lines
Can't we do this already today by defining a simple errors/metadata file in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? This will create a record on dbus in phosphor-logging.
I think the original concern was with supporting on the order of 10,000 log entries, having this on d-bus seemed impractical. Also the free log rotation the journal provides is useful. Now modifying the logging::report<...> to conditionally log to the journal seems realistic.
My intention was not to re-implement the logging, my intention was to extend/use the existing design which we already have it below.
https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
Quoted 8 lines
I was trying not to bring the Redfish specific stuff in each individual repo, instead each transport can listen for Dbus events and write to the journal which goes to their app specific file.
Your point is valid that we would be polluting the journal if each and every transport start writing as per their needs.
***** As per the Redfish one of the requirement is we need the log for most of the Dbus Property update/interface added as they are mapped to some Redfish Resource and the bmcweb has to send the Resource updated/modified signal to the Redfish client. ******
Why do we want to log on D-Bus property updates? This seems like it will be too noisy for the EventLog.
Quoted 25 lines
As we are in agreement that we want to use the journal for persistence and log rotate feature.
We have two options:
1) Each transport interface listens for the Dbus signals and write
it to their app specific file.
2) Each openbmc repo must uselog::reportfor each D-bus property
update/ interface added.
To make this happen, we need the following
* common message format which can be used by the all other
application (SEL,Redfish) etc.
* Transport application will read this common message from the
journal and use it as per their needs.Option no 2: It is ideal solution but that requires good amount of work as
either each and every D-bus repo should write to the
journal for any property update/interface added.
or
Should SDbusplus do this as each and every application
uses the sdbusplus framework to manage the dbus objectsPlease let me know your thoughts.
Ratan
Quoted 54 lines · Jason M
Quoted 47 lines · Ratan Gupta
Hi James,Pattrick.
Thanks for giving the feedbackQuoted 44 lines · James Feist
Quoted 38 lines · Patrick Williams
Quoted 9 lines · James Feist
Quoted 7 lines · Ratan Gupta
eg: sd_journal_send("MESSAGE=%s", "Account Modified", "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s", "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>", "REDFISH_RESOURCE_TYPE=ComputerSystem" "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);Quoted 6 lines
If we're going to go down the path of re-implementing logging, I think the goal should be to stop logging things in the Journal that are Redfish specific, and instead log them in some generic format that phosphor logging controls the map. Right now we are bifurcated because the dbus-event-logs, SEL, PEL, and Redfish are all using different methods of logging, that play to their own set of rules.
Absolutely agree with you here. There is zero reason that applications should start logging specially formed messages with REDFISH_* meta-data. We shouldn't have any applications explicitly know about Redfish except the Redfish providers themselves. This is no different from IPMI, PLDM, or any other external interface.
The kind of information presented here as being interesting to expose via Redfish is equally as interesting to me to be able to add to one of our 'FFDC dumps', which could be used for security / forensic work.
Quoted 10 lines
Most repos use
phosphor-logging, so instead of creating yet another daemon, if we added support to create a 'System Level' or 'External User' log that has predefined dictionaries of required and optional keys, something like phosphor-dbus-interfaces, we might be able to drop some of these transport specific logs, and have the transports based on the same database (the journal). Then each transport could filter these based on journal entry type, and transform them into the correct log for that transport. I think adding more Redfish specifics into the logs hinders those who do not want Redfish in their systems.Quoted 4 lines
Can't we do this already today by defining a simple errors/metadata file in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? This will create a record on dbus in phosphor-logging.
I think the original concern was with supporting on the order of 10,000 log entries, having this on d-bus seemed impractical. Also the free log rotation the journal provides is useful. Now modifying the logging::report<...> to conditionally log to the journal seems realistic.
Quoted 8 lines
Quoted 7 lines
My intention was not to re-implement the logging, my intention was to extend/use the existing design which we already have it below.
https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md I was trying not to bring the Redfish specific stuff in each individual repo, instead each transport can listen for Dbus events and write to the journal which goes to their app specific file.
Your point is valid that we would be polluting the journal if each and every transport start writing as per their needs.
***** As per the Redfish one of the requirement is we need the log for most of the Dbus Property update/interface added as they
are mapped to some Redfish Resource and the bmcweb has to send the Resource updated/modified signal to the
Redfish client. ******Why do we want to log on D-Bus property updates? This seems like it will be too noisy for the EventLog.
I don’t think it would be all, just the one’s the different transport layers need.
So to get my head straight on this, there is a commit in state-manager to log the redfish events to the journal:
https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-state-manager/+/33494
With what Ratan proposed above, we have two options?
1) A separate application that listens for the chassis properties to change and logs those events to the journal in the format the transport layer wants
2) Code within phosphor-state-manager that logs a generic event to the journal which then can be post processed by the different transport layer code.
Quoted 14 lines
Quoted 14 lines
As we are in agreement that we want to use the journal for persistence and log rotate feature.
We have two options:
1) Each transport interface listens for the Dbus signals and write it to their app specific file.
2) Each openbmc repo must uselog::reportfor each D-bus property update/ interface added.
To make this happen, we need the following
* common message format which can be used by the all other application (SEL,Redfish) etc.
* Transport application will read this common message from the journal and use it as per their needs.
Option no 2: It is ideal solution but that requires good amount of work as
either each and every D-bus repo should write to the journal for any property update/interface added.
or
Should SDbusplus do this as each and every application uses the sdbusplus framework to manage the dbus objects
Please let me know your thoughts.
Ratan
Quoted 10 lines · Ratan Gupta
Hi James,Pattrick.
Quoted 8 lines
Quoted 4 lines
Can't we do this already today by defining a simple errors/metadata file in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? This will create a record on dbus in phosphor-logging.
I think the original concern was with supporting on the order of 10,000 log entries, having this on d-bus seemed impractical. Also the free log rotation the journal provides is useful. Now modifying the logging::report<...> to conditionally log to the journal seems realistic.
Quoted 8 lines
My intention was not to re-implement the logging, my intention was to extend/use the existing design which we already have it below.
https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
I was trying not to bring the Redfish specific stuff in each individual repo, instead each transport can listen for
Dbus events and write to the journal which goes to their app specific file.
Good. This wasn't clear from the earlier email. Thanks.
Quoted 2 lines
As we are in agreement that we want to use the journal for persistence and log rotate feature.
I'm not convinced there is agreement on this. There has been disagreement about even using the journal for phosphor-logging use since the beginning and I suspect there would be less agreement on another application using it as its own IPC mechanism.
Just because a hammer can be used to insert a nail into a board doesn't mean you use it insert any pointy thing into a flat thing. [ Just because the journal provides log rotation and persistance doesn't mean you should use it for every feature needing log rotation and persistance. ]
Quoted 5 lines
***** As per the Redfish one of the requirement is we need the log for most of the Dbus Property update/interface added as they are mapped to some Redfish Resource and the bmcweb has to send the Resource updated/modified signal to the
Redfish client. ******
I don't know Redfish well, so bear with me if there is something obvious I'm missing. But, the first part of this "requirement" doesn't seem to follow from the second part of the "requirement" to me.
Sending a signal of a property changing to the Redfish clients is straight-forwawrd; Redfish should subscribe to all the appropriate dbus-events. I don't understand how this implies any sort of logging. Where does the logging part of this requirement come from?
Quoted 5 lines
We have two options:
1) Each transport interface listens for the Dbus signals and write
it to their app specific file.
2) Each openbmc repo must uselog::reportfor each D-bus property
update/ interface added.
#2 is absolutely unworkable on the surface to me. log::report is to create a error entry (xyz.openbmc_project.Logging.Entry), which creates a dbus-object, which would cause log::report to be called, which creates a dbus-object, which ...
Even if what you meant was something like logging::log<info>, this seems pretty heavy. I'm not sure this is something that can be inserted into sdbusplus, especially for the ASIO-based object servers, because in many cases applications register their own callback. For the sdbus++ generated server bindings we could squeeze it in. But, what you're proposing here is essentially a "journal-as-dbus-monitor". We'd certainly need to make some measurements on how many kB/s worth of journal entries this would create because I suspect we could end up burning out a NAND flash with as many writes as that would induce.
Signature · 1 line
Patrick Williams
Quoted 29 lines · Patrick Williams
Quoted 10 lines · Ratan Gupta
Hi James,Pattrick.
Quoted 8 lines
Quoted 4 lines
Can't we do this already today by defining a simple errors/metadata file in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? This will create a record on dbus in phosphor-logging.
I think the original concern was with supporting on the order of 10,000 log entries, having this on d-bus seemed impractical. Also the free log rotation the journal provides is useful. Now modifying the logging::report<...> to conditionally log to the journal seems realistic.
Quoted 8 lines
My intention was not to re-implement the logging, my intention was to extend/use the existing design which we already have it below.
https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
I was trying not to bring the Redfish specific stuff in each individual repo, instead each transport can listen for
Dbus events and write to the journal which goes to their app specific file.Good. This wasn't clear from the earlier email. Thanks.
Quoted 2 lines
As we are in agreement that we want to use the journal for persistence and log rotate feature.
I'm not convinced there is agreement on this. There has been disagreement about even using the journal for phosphor-logging use since the beginning and I suspect there would be less agreement on another application using it as its own IPC mechanism.
Quoted 12 lines
Just because a hammer can be used to insert a nail into a board doesn't mean you use it insert any pointy thing into a flat thing. [ Just because the journal provides log rotation and persistance doesn't mean you should use it for every feature needing log rotation and persistance. ]
Quoted 5 lines
***** As per the Redfish one of the requirement is we need the log for most of the Dbus Property update/interface added as they are mapped to some Redfish Resource and the bmcweb has to send the Resource updated/modified signal to the
Redfish client. ******
Jaosn: You asked the following query in other thread /*"Why do we want to log on D-Bus property updates? This seems like it will be too noisy for the EventLog*"/
Eg: Client is interested for an event when ever there is any user add/delete or network configuration change or there is a log entry resource gets created,To handle this request the flow would be
Redfish Client subscribe for "ResourceType" eg: "EthernetService,AccountService,LogService" with subordinate resources property as truewhich means the Client is looking for updates on the subscribed resources and the subordinates resource, These redfish resources(EthernetInterface, IP address, ManagerAccount, AccountService) would be mapped to some D-bus Resources, hence some application/bmcweb would monitor the Dbus signals on the interested Dbus objects and send the Redfish event to the subscribed client.
Apparo: Please correct me if I am missing something.
Quoted 8 lines
I don't know Redfish well, so bear with me if there is something obvious I'm missing. But, the first part of this "requirement" doesn't seem to follow from the second part of the "requirement" to me.
Sending a signal of a property changing to the Redfish clients is straight-forwawrd; Redfish should subscribe to all the appropriate dbus-events. I don't understand how this implies any sort of logging. Where does the logging part of this requirement come from?
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749/16/designs/redfish-eventservice.md#474
While I am reading the redfish spec(https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.11.0.pdf) , it is not clear that the events need to be persisted.I will ask in the DMTF for the persistence of the events.
Quoted 10 lines
Quoted 5 lines
We have two options:
1) Each transport interface listens for the Dbus signals and write
it to their app specific file.
2) Each openbmc repo must uselog::reportfor each D-bus property
update/ interface added.#2 is absolutely unworkable on the surface to me.
log::reportis to create a error entry (xyz.openbmc_project.Logging.Entry), which creates a dbus-object, which would causelog::reportto be called, which creates a dbus-object, which ...
Quoted 9 lines
Even if what you meant was something like logging::log<info>, this seems pretty heavy. I'm not sure this is something that can be inserted into sdbusplus, especially for the ASIO-based object servers, because in many cases applications register their own callback. For the sdbus++ generated server bindings we could squeeze it in. But, what you're proposing here is essentially a "journal-as-dbus-monitor". We'd certainly need to make some measurements on how many kB/s worth of journal entries this would create because I suspect we could end up burning out a NAND flash with as many writes as that would induce.
I would respond on the same once my query gets answered from DMTF. If my query gets answered yes then we have to write on flash but let's wait for it,
Quoted 44 lines · Ratan Gupta
Quoted 29 lines · Patrick Williams
Quoted 10 lines · Ratan Gupta
Hi James,Pattrick.
Quoted 8 lines
Quoted 4 lines
Can't we do this already today by defining a simple errors/metadata file in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? This will create a record on dbus in phosphor-logging.
I think the original concern was with supporting on the order of 10,000 log entries, having this on d-bus seemed impractical. Also the free log rotation the journal provides is useful. Now modifying the logging::report<...> to conditionally log to the journal seems realistic.
Quoted 8 lines
My intention was not to re-implement the logging, my intention was to extend/use the existing design which we already have it below.
https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
I was trying not to bring the Redfish specific stuff in each individual repo, instead each transport can listen for
Dbus events and write to the journal which goes to their app specific file.Good. This wasn't clear from the earlier email. Thanks.
Quoted 2 lines
As we are in agreement that we want to use the journal for persistence and log rotate feature.
I'm not convinced there is agreement on this. There has been disagreement about even using the journal for phosphor-logging use since the beginning and I suspect there would be less agreement on another application using it as its own IPC mechanism.
Quoted 12 lines
Just because a hammer can be used to insert a nail into a board doesn't mean you use it insert any pointy thing into a flat thing. [ Just because the journal provides log rotation and persistance doesn't mean you should use it for every feature needing log rotation and persistance. ]
Quoted 5 lines
***** As per the Redfish one of the requirement is we need the log for most of the Dbus Property update/interface added as they are mapped to some Redfish Resource and the bmcweb has to send the Resource updated/modified signal to the
Redfish client. ******
Quoted 3 lines
Jaosn: You asked the following query in other thread /*"Why do we want to log on D-Bus property updates? This seems like it will be too noisy for the EventLog*"/
Eg: Client is interested for an event when ever there is any user add/delete or network configuration change or there is a log entry resource gets created,To handle this request the flow would be
My understanding is for the first iteration we are only providing notifications for event messages. So, if it isn't in the message registry, then you cannot register for that event?
It seems like the full resource change event notification will be a major feature and should probably get a dedicated design.
Quoted 39 lines
Redfish Client subscribe for "ResourceType" eg: "EthernetService,AccountService,LogService" with subordinate resources property as truewhich means the Client is looking for updates on the subscribed resources and the subordinates resource, These redfish
resources(EthernetInterface, IP address, ManagerAccount, AccountService)would be mapped to some D-bus Resources, hence some application/bmcweb would monitor the Dbus signals on the interested Dbus objects and send the Redfish event to the subscribed client.Apparo: Please correct me if I am missing something.
Quoted 8 lines
I don't know Redfish well, so bear with me if there is something obvious I'm missing. But, the first part of this "requirement" doesn't seem to follow from the second part of the "requirement" to me.
Sending a signal of a property changing to the Redfish clients is straight-forwawrd; Redfish should subscribe to all the appropriate dbus-events. I don't understand how this implies any sort of logging. Where does the logging part of this requirement come from?
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749/16/designs/redfish-eventservice.md#474
While I am reading the redfish spec(https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.11.0.pdf) , it is not clear that the events need to be persisted.I will ask in the DMTF for the persistence of the events.
Quoted 10 lines
Quoted 5 lines
We have two options:
1) Each transport interface listens for the Dbus signals and write
it to their app specific file.
2) Each openbmc repo must uselog::reportfor each D-bus property
update/ interface added.#2 is absolutely unworkable on the surface to me.
log::reportis to create a error entry (xyz.openbmc_project.Logging.Entry), which creates a dbus-object, which would causelog::reportto be called, which creates a dbus-object, which ...Quoted 9 lines
Even if what you meant was something like logging::log<info>, this seems pretty heavy. I'm not sure this is something that can be inserted into sdbusplus, especially for the ASIO-based object servers, because in many cases applications register their own callback. For the sdbus++ generated server bindings we could squeeze it in. But, what you're proposing here is essentially a "journal-as-dbus-monitor". We'd certainly need to make some measurements on how many kB/s worth of journal entries this would create because I suspect we could end up burning out a NAND flash with as many writes as that would induce.
I would respond on the same once my query gets answered from DMTF. If my query gets answered yes then we have to write on flash but let's wait for it,
Quoted 57 lines · Ratan Gupta
Quoted 29 lines · Patrick Williams
Quoted 10 lines · Ratan Gupta
Hi James,Pattrick.
Quoted 8 lines
Quoted 4 lines
Can't we do this already today by defining a simple errors/metadata file in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? This will create a record on dbus in phosphor-logging.
I think the original concern was with supporting on the order of 10,000 log entries, having this on d-bus seemed impractical. Also the free log rotation the journal provides is useful. Now modifying the logging::report<...> to conditionally log to the journal seems realistic.
Quoted 8 lines
My intention was not to re-implement the logging, my intention was to extend/use the existing design which we already have it below.
https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
I was trying not to bring the Redfish specific stuff in each individual repo, instead each transport can listen for
Dbus events and write to the journal which goes to their app specific file.Good. This wasn't clear from the earlier email. Thanks.
Quoted 2 lines
As we are in agreement that we want to use the journal for persistence and log rotate feature.
I'm not convinced there is agreement on this. There has been disagreement about even using the journal for phosphor-logging use since the beginning and I suspect there would be less agreement on another application using it as its own IPC mechanism.
Quoted 12 lines
Just because a hammer can be used to insert a nail into a board doesn't mean you use it insert any pointy thing into a flat thing. [ Just because the journal provides log rotation and persistance doesn't mean you should use it for every feature needing log rotation and persistance. ]
Quoted 5 lines
***** As per the Redfish one of the requirement is we need the log for most of the Dbus Property update/interface added as they are mapped to some Redfish Resource and the bmcweb has to send the Resource updated/modified signal to the
Redfish client. ******Jaosn: You asked the following query in other thread /*"Why do we want to log on D-Bus property updates? This seems like it will be too noisy for the EventLog*"/
Eg: Client is interested for an event when ever there is any user add/delete or network configuration change or there is a log entry resource gets created,To handle this request the flow would be
Redfish Client subscribe for "ResourceType" eg: "EthernetService,AccountService,LogService" with subordinate resources property as truewhich means the Client is looking for updates on the subscribed resources and the subordinates resource, These redfish
resources(EthernetInterface, IP address, ManagerAccount, AccountService)would be mapped to some D-bus Resources, hence some application/bmcweb would monitor the Dbus signals on the interested Dbus objects and send the Redfish event to the subscribed client.Apparo: Please correct me if I am missing something.
Quoted 4 lines
I don't know Redfish well, so bear with me if there is something obvious I'm missing. But, the first part of this "requirement" doesn't seem to follow from the second part of the "requirement" to me.
Quoted 7 lines
Quoted 4 lines
Sending a signal of a property changing to the Redfish clients is straight-forwawrd; Redfish should subscribe to all the appropriate dbus-events. I don't understand how this implies any sort of logging. Where does the logging part of this requirement come from?
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749/16/designs/redfish-eventservice.md#474
While I am reading the redfish spec(https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.11.0.pdf) , it is not clear that the events need to be persisted.I will ask in the DMTF for the persistence of the events.
I have got an update from the DMTF that persistence is not required to be implemented for the events(https://redfish.dmtf.org/schemas/v1/Event.v1_5_0.json), which implies we don't need to log the events in Journal.
More info(https://github.com/DMTF/Redfish/issues/3646)
Now updating the proposal where redfish client subscribed for Resource Type based events
1) Client would subscribe for the Redfish Resource(eg:ManagerAccount) to receive events like Account add/delete/modify Hence need for mapping from (RedfishResource to Dbus Resource)
2) Have the mapping info from Redfish resources to DBUS Resources (Some JSon file may have this mapping)
2) Have the reverse mapping from DBUS Resources to Redfish Resources
* This is needed to send the Redfish event if there is any changes in the corresponding D-bus resources. eg BMC state change/network change etc.
3) bmcweb would monitor the DBUS events
4) Get the Redfish Path from the Mapping(2) and send the Redfish event
5) Bmcweb would buffer N number of events that can be configurable by redfish client. Buffer would get cleaned up in case of bmcweb restart or BMC reset.
James, Apparao: Please let me know if there is any more concern with this approach.
Quoted 23 lines
Quoted 11 lines
Quoted 6 lines
We have two options:
1) Each transport interface listens for the Dbus signals and write
it to their app specific file.
2) Each openbmc repo must uselog::reportfor each D-bus property
update/ interface added.#2 is absolutely unworkable on the surface to me.
log::reportis to create a error entry (xyz.openbmc_project.Logging.Entry), which creates a dbus-object, which would causelog::reportto be called, which creates a dbus-object, which ...Quoted 9 lines
Even if what you meant was something like logging::log<info>, this seems pretty heavy. I'm not sure this is something that can be inserted into sdbusplus, especially for the ASIO-based object servers, because in many cases applications register their own callback. For the sdbus++ generated server bindings we could squeeze it in. But, what you're proposing here is essentially a "journal-as-dbus-monitor". We'd certainly need to make some measurements on how many kB/s worth of journal entries this would create because I suspect we could end up burning out a NAND flash with as many writes as that would induce.
I would respond on the same once my query gets answered from DMTF. If my query gets answered yes then we have to write on flash but let's wait for it,
Quoted 10 lines
I have got an update from the DMTF that persistence is not required to be implemented for the events(https://redfish.dmtf.org/schemas/v1/Event.v1_5_0.json), which implies we don't need to log the events in Journal.
More info(https://github.com/DMTF/Redfish/issues/3646)
Now updating the proposal where redfish client subscribed for Resource Type based events
1) Client would subscribe for the Redfish Resource(eg:ManagerAccount) to receive events like Account add/delete/modify Hence need for mapping from (RedfishResource to Dbus Resource)2) Have the mapping info from Redfish resources to DBUS Resources (Some JSon file may have this mapping)
This assumes the mapping is static, which on many systems it isn't, right? I think this needs to be developed to see what it would be like.
Quoted 7 lines
2) Have the reverse mapping from DBUS Resources to Redfish Resources
* This is needed to send the Redfish event if there is any changes
in the corresponding D-bus resources.
eg BMC state change/network change etc.3) bmcweb would monitor the DBUS events
Which events specifically? It seems like it would be monitoring lots of events in this proposal. Some examples could be useful.
Quoted 5 lines
4) Get the Redfish Path from the Mapping(2) and send the Redfish event
5) Bmcweb would buffer N number of events that can be configurable by redfish client. Buffer would get cleaned up in case of bmcweb restart or BMC reset.
Can you please push a patch to the event service design doc with the change to the design you'd like? Some examples in the design would help with understanding the proposal. This thread is quite long and hard to follow for someone not involved, it'd be good to document the proposed changes.
https://github.com/openbmc/docs/blob/master/designs/redfish-eventservice.md
Quoted 6 lines · James Feist
Quoted 2 lines
2) Have the mapping info from Redfish resources to DBUS Resources (Some JSon file may have this mapping)
This assumes the mapping is static, which on many systems it isn't, right? I think this needs to be developed to see what it would be like.
Don't we, in effect, have the mapping from Redfish to dbus by nature of the Redfish providers that create their content from dbus objects? Is there any way to reuse this information to create the corresponding Redfish events?
Signature · 1 line
Patrick Williams
Quoted 4 lines · James Feist
Quoted 1 line
3) bmcweb would monitor the DBUS events
Which events specifically? It seems like it would be monitoring lots of events in this proposal.
Just curious what is the concern around monitoring lots of events?
Quoted 8 lines · Brad Bishop
Quoted 4 lines · James Feist
Quoted 1 line
3) bmcweb would monitor the DBUS events
Which events specifically? It seems like it would be monitoring lots of events in this proposal.
Just curious what is the concern around monitoring lots of events?
Performance as always, depending on the number of matches needed. Although a POC could prove this not an issue.
Quoted 2 lines · James Feist
This thread is quite long and hard to follow for someone not involved, it'd be good to document the proposed changes.
You are right of course that this thread has become hard to follow. I think careful selection of reply context really would have helped.
FWIW I have asked the team here at IBM to hash out designs to some level of consensus here on the list before making design submissions to gerrit because IMO reviews of half baked designs in gerrit are even harder to follow than email threads like this one 🙂
Quoted 4 lines · Brad Bishop
Quoted 2 lines · James Feist
This thread is quite long and hard to follow for someone not involved, it'd be good to document the proposed changes.
Quoted 8 lines
You are right of course that this thread has become hard to follow. I think careful selection of reply context really would have helped.
FWIW I have asked the team here at IBM to hash out designs to some level of consensus here on the list before making design submissions to gerrit because IMO reviews of half baked designs in gerrit are even harder to follow than email threads like this one 🙂
That's good. Was just hoping that the 'final design' wasn't this email, as it would be difficult to see we matched a design.
Quoted 6 lines · Ratan Gupta
1) Client would subscribe for the Redfish Resource(eg:ManagerAccount) to receive events like Account add/delete/modify. Hence need for mapping from (RedfishResource to Dbus Resource)
2) Have the mapping info from Redfish resources to DBUS Resources (Some JSon file may have this mapping)
It probably doesn't make sense to have any json files. The application logic itself does the mapping.
Quoted 8 lines
3) Have the reverse mapping from DBUS Resources to Redfish Resources This is needed to send the Redfish event if there is any changes in the corresponding D-bus resources. eg BMC state change/network change etc.
4) bmcweb would monitor the DBUS events
5) Get the Redfish Path from the Mapping(3) and send the Redfish event
Let me try to restate 3-5. We already have application logic in bmcweb that maps redfish resources to dbus interfaces, triggered by redfish client requests (like a GET or PATCH). The proposal here I think is to implement the reverse - add application logic in bmcweb that maps dbus interfaces to redfish resources, triggered by dbus signals. I think this is more or less what was suggested by Patrick?
Quoted 5 lines
Don't we, in effect, have the mapping from Redfish to dbus by nature of the Redfish providers that create their content from dbus objects?
James, Apparao: Please let me know if there is any more concern with this approach.
Ratan you had 2x #2s in the flow above, which I fixed. Please let me know if I got it wrong.
The main concerns with this approach that I have heard are too many signal matches and complexity of the logic implementing the reverse map. I can definitely see how the logic will get complicated given our current dbus data model.
One idea floating around to address these is inventing a journal metadata scheme that is management interface agnostic. I understand the motivation behind that - it is much simpler for an application to slide a single journal entry into the journal with all the metadata needed to generate events, than it is for an application to snoop multiple signals off dbus and pull events out of that.
But I wonder if inventing a management interface agnostic scheme for adding events to the journal is really just inventing a new data model for how we represent things in a server - e.g. are we just working around our dbus data model? Maybe we should fix it instead, so that it isn't so difficult for applications to use it? With that said I don't know how to do this and I could use more concrete details on which areas of the data model make it hard to consume signals. Does anyone have any ideas or examples?
Quoted 7 lines · Brad Bishop
One idea floating around to address these is inventing a journal metadata scheme that is management interface agnostic. I understand the motivation behind that - it is much simpler for an application to slide a single journal entry into the journal with all the metadata needed to generate events, than it is for an application to snoop multiple signals off dbus and pull events out of that.
Quoted 9 lines
But I wonder if inventing a management interface agnostic scheme for adding events to the journal is really just inventing a new data model for how we represent things in a server - e.g. are we just working around our dbus data model? Maybe we should fix it instead, so that it isn't so difficult for applications to use it? With that said I don't know how to do this and I could use more concrete details on which areas of the data model make it hard to consume signals. Does anyone have any ideas or examples?
On this, I think we may want to separate logging vs. eventing both in this feature discussion and in the tools we want to use.
When we were talking about logging, I think the journal made sense since it is designed for logging and has benefits around that usage. However, I agree that it doesn't seem like the right tool for sending and receiving events and signals and that D-Bus sounds like the right tool for that.
I think I'm still a little confused at the scope. My understanding was that this initial design for EventService was only for monitoring event messages and not resources in general. It seems like it may not make sense to try to use the same tools and approach for both message monitoring and resource monitoring? Do we need to treat them separately for now to simplify the discussion?
Hi Jason,james
Quoted 20 lines · Jason M
Quoted 7 lines · Brad Bishop
One idea floating around to address these is inventing a journal metadata scheme that is management interface agnostic. I understand the motivation behind that - it is much simpler for an application to slide a single journal entry into the journal with all the metadata needed to generate events, than it is for an application to snoop multiple signals off dbus and pull events out of that.
Quoted 9 lines
But I wonder if inventing a management interface agnostic scheme for adding events to the journal is really just inventing a new data model for how we represent things in a server - e.g. are we just working around our dbus data model? Maybe we should fix it instead, so that it isn't so difficult for applications to use it? With that said I don't know how to do this and I could use more concrete details on which areas of the data model make it hard to consume signals. Does anyone have any ideas or examples?
Quoted 5 lines
On this, I think we may want to separate logging vs. eventing both in this feature discussion and in the tools we want to use.
When we were talking about logging, I think the journal made sense since it is designed for logging and has benefits around that usage. However, I agree that it doesn't seem like the right tool for sending and receiving events and signals and that D-Bus sounds like the right tool for that.
I think I'm still a little confused at the scope. My understanding was that this initial design for EventService was only for monitoring event messages and not resources in general. It seems like it may not make sense to try to use the same tools and approach for both message monitoring and resource monitoring? Do we need to treat them separately for now to simplify the discussion?
Jason, When you say event messages? What do you mean, Do you mean to say "/redfish/v1/Systems/system/Logservices/eventlog"?
If yes then this should also go as Resource Event, When ever any log entry gets created under System Log (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would notify to the Redfish client saying that "ResourceCreated" with the URL of the Resource.
After recieving this event Redfish client will do a GET request on the URL(retrieved as part of event) to get the content of the log.
I would be coming up with few design approaches and downside with each approach to take it to conclusion.
Ratan
Quoted 20 lines · Jason M
Quoted 7 lines · Brad Bishop
One idea floating around to address these is inventing a journal metadata scheme that is management interface agnostic. I understand the motivation behind that - it is much simpler for an application to slide a single journal entry into the journal with all the metadata needed to generate events, than it is for an application to snoop multiple signals off dbus and pull events out of that.
Quoted 9 lines
But I wonder if inventing a management interface agnostic scheme for adding events to the journal is really just inventing a new data model for how we represent things in a server - e.g. are we just working around our dbus data model? Maybe we should fix it instead, so that it isn't so difficult for applications to use it? With that said I don't know how to do this and I could use more concrete details on which areas of the data model make it hard to consume signals. Does anyone have any ideas or examples?
Quoted 5 lines
On this, I think we may want to separate logging vs. eventing both in this feature discussion and in the tools we want to use.
When we were talking about logging, I think the journal made sense since it is designed for logging and has benefits around that usage. However, I agree that it doesn't seem like the right tool for sending and receiving events and signals and that D-Bus sounds like the right tool for that.
I think I'm still a little confused at the scope. My understanding was that this initial design for EventService was only for monitoring event messages and not resources in general. It seems like it may not make sense to try to use the same tools and approach for both message monitoring and resource monitoring? Do we need to treat them separately for now to simplify the discussion?
Jason, When you say event messages? What do you mean, Do you mean to say "/redfish/v1/Systems/system/Logservices/eventlog"?
If yes then this should also go as Resource Event, When ever any log entry gets created under System Log (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would notify to the Redfish client saying that "ResourceCreated" with the URL of the Resource.
After receiving this event Redfish client will do a GET request on the URL(retrieved as part of event) to get the content of the log.
This will become generic infra for all types of events.
I would be coming up with few design approaches and downside with each approach to take it to conclusion.
Ratan
Quoted 22 lines · Ratan Gupta
Quoted 20 lines · Jason M
Quoted 16 lines · Brad Bishop
One idea floating around to address these is inventing a journal metadata scheme that is management interface agnostic. I understand the motivation behind that - it is much simpler for an application to slide a single journal entry into the journal with all the metadata needed to generate events, than it is for an application to snoop multiple signals off dbus and pull events out of that.
But I wonder if inventing a management interface agnostic scheme for adding events to the journal is really just inventing a new data model for how we represent things in a server - e.g. are we just working around our dbus data model? Maybe we should fix it instead, so that it isn't so difficult for applications to use it? With that said I don't know how to do this and I could use more concrete details on which areas of the data model make it hard to consume signals. Does anyone have any ideas or examples?
Quoted 7 lines
Quoted 5 lines
On this, I think we may want to separate logging vs. eventing both in this feature discussion and in the tools we want to use.
When we were talking about logging, I think the journal made sense since it is designed for logging and has benefits around that usage. However, I agree that it doesn't seem like the right tool for sending and receiving events and signals and that D-Bus sounds like the right tool for that.
I think I'm still a little confused at the scope. My understanding was that this initial design for EventService was only for monitoring event messages and not resources in general. It seems like it may not make sense to try to use the same tools and approach for both message monitoring and resource monitoring? Do we need to treat them separately for now to simplify the discussion?
Jason, When you say event messages? What do you mean, Do you mean to say "/redfish/v1/Systems/system/Logservices/eventlog"? >
If yes then this should also go as Resource Event, When ever any log entry gets created under System Log (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would notify to the Redfish client saying that "ResourceCreated" with the URL of the Resource.
Yes, new entries under "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you could register for specific MessageIDs, so it's not just a generic "new resource" event like others would be.
Quoted 4 lines
After receiving this event Redfish client will do a GET request on the URL(retrieved as part of event) to get the content of the log.
This will become generic infra for all types of events.
What I'm saying is I don't know if there is a good generic solution to cover both the EventLog and all other resources. I believe the current EventService implementation was designed only for EventLog and may not work well for generic resource events.
Quoted 2 lines
I would be coming up with few design approaches and downside with each approach to take it to conclusion.
Thanks! What I'm proposing is that we clarify or possibly separate the discussions about EventLog vs. generic resources to avoid confusion and come up with the right solutions for each.
Quoted 3 lines
Ratan
Quoted 25 lines · Jason M
Quoted 22 lines · Ratan Gupta
Quoted 20 lines · Jason M
Quoted 16 lines · Brad Bishop
One idea floating around to address these is inventing a journal metadata scheme that is management interface agnostic. I understand the motivation behind that - it is much simpler for an application to slide a single journal entry into the journal with all the metadata needed to generate events, than it is for an application to snoop multiple signals off dbus and pull events out of that.
But I wonder if inventing a management interface agnostic scheme for adding events to the journal is really just inventing a new data model for how we represent things in a server - e.g. are we just working around our dbus data model? Maybe we should fix it instead, so that it isn't so difficult for applications to use it? With that said I don't know how to do this and I could use more concrete details on which areas of the data model make it hard to consume signals. Does anyone have any ideas or examples?
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
On this, I think we may want to separate logging vs. eventing both in this feature discussion and in the tools we want to use.
When we were talking about logging, I think the journal made sense since it is designed for logging and has benefits around that usage. However, I agree that it doesn't seem like the right tool for sending and receiving events and signals and that D-Bus sounds like the right tool for that.
I think I'm still a little confused at the scope. My understanding was that this initial design for EventService was only for monitoring event messages and not resources in general. It seems like it may not make sense to try to use the same tools and approach for both message monitoring and resource monitoring? Do we need to treat them separately for now to simplify the discussion?
Jason, When you say event messages? What do you mean, Do you mean to say "/redfish/v1/Systems/system/Logservices/eventlog"? >
If yes then this should also go as Resource Event, When ever any log entry gets created under System Log (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would notify to the Redfish client saying that "ResourceCreated" with the URL of the Resource.Yes, new entries under "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you could register for specific MessageIDs, so it's not just a generic "new resource" event like others would be.
Can we register for MessageID? I thought client can register for whole registry not a specific Message ID.
Quoted 7 lines
Quoted 6 lines
After receiving this event Redfish client will do a GET request on the URL(retrieved as part of event) to get the content of the log.
This will become generic infra for all types of events.
What I'm saying is I don't know if there is a good generic solution to cover both the EventLog and all other resources. I believe the current EventService implementation was designed only for EventLog and may not work well for generic resource events.
Can you get me the example payload for EventLog which is going to be sent with the current design? I am not sure how the eventlog and other resources are different.
For eventLogs also we have the associated D-bus objects(/xyz/openbmc_project/logging,/xyz/openbmc_project/dump etc)
Quoted 8 lines
Quoted 3 lines
I would be coming up with few design approaches and downside with each approach to take it to conclusion.
Thanks! What I'm proposing is that we clarify or possibly separate the discussions about EventLog vs. generic resources to avoid confusion and come up with the right solutions for each.
Quoted 3 lines
Ratan
Quoted 7 lines · Ratan Gupta
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
I think I'm still a little confused at the scope. My understanding was that this initial design for EventService was only for monitoring event messages and not resources in general. It seems like it may not make sense to try to use the same tools and approach for both message monitoring and resource monitoring? Do we need to treat them separately for now to simplify the discussion?
Jason, When you say event messages? What do you mean, Do you mean to say "/redfish/v1/Systems/system/Logservices/eventlog"? >
If yes then this should also go as Resource Event, When ever any log entry gets created under System Log (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would notify to the Redfish client saying that "ResourceCreated" with the URL of the Resource.Yes, new entries under "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you could register for specific MessageIDs, so it's not just a generic "new resource" event like others would be.
Can we register for MessageID? I thought client can register for whole registry not a specific Message ID.
I don't really know. I thought that's what the current implementation allowed, but I don't know for sure if it can or should.
Quoted 10 lines
Quoted 7 lines
Quoted 6 lines
After receiving this event Redfish client will do a GET request on the URL(retrieved as part of event) to get the content of the log.
This will become generic infra for all types of events.
What I'm saying is I don't know if there is a good generic solution to cover both the EventLog and all other resources. I believe the current EventService implementation was designed only for EventLog and may not work well for generic resource events.
Can you get me the example payload for EventLog which is going to be sent with the current design? I am not sure how the eventlog and other resources are different.
This is based on the assumption that for a LogService, you can register for a MessageId. If this is not possible, then they might be treated the same.
Quoted 2 lines
For eventLogs also we have the associated D-bus
objects(/xyz/openbmc_project/logging,/xyz/openbmc_project/dump etc)
For Intel platforms, we don't use /xyz/openbmc_project/logging, so we don't have D-Bus objects associated with each EventLog LogEntry. We use rsyslog to create a file that contains many LogEntries.
However, as an unrelated side-thought: linking logging to /xyz/openbmc_project/dump made me wonder if there is a possible solution to the logging issue if we treat /xyz/openbmc_project/logging like /xyz/openbmc_project/dump and place a pointer to the log in the D-Bus object instead of the log itself?
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
I would be coming up with few design approaches and downside with each approach to take it to conclusion.
Thanks! What I'm proposing is that we clarify or possibly separate the discussions about EventLog vs. generic resources to avoid confusion and come up with the right solutions for each.
Quoted 3 lines
Ratan
Hi James,
As you asked for the examples in the thread(https://lists.ozlabs.org/pipermail/openbmc/2020-June/022125.html), I have created the mapping at the following location.
https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2
1) Map1: Mapping between redfish resources to Dbus resources
2) Map2: Mapping between redfish resource types to the ineterested Dbus interfaces
3) Map3: Mapping between Dbus resources to redfish resources
I tried to cover the following scenario in the above mapping.
* Redfish resource is mapped to multiple Dbus Resources
* Redfish Property is mapped to single Dbus property
* Redfish Property(complex property) is mapped to multiple dbus property.
* Same type of Redfish Resources are mapped to different Dbus Resources
* Redfish node url having multiple regex : Yet to take a look on this.
Flow would be like as below
=> In bmcweb each Redfish node represents to a Redfish Resource.
=> Each node will be having it's own mapping between Redfish properties and the Dbus Resources.
=> Some code on bmcweb will walkthrough on each node during startup , get this mapping from each node and generate two mappings
1) Reverse mapping (Dbus Resource to Redfish Resource)(MAP3) and
2) mapping between Resource Types to the interested Dbus interfaces(MAP2)
=> To start with we will support few resource types and then scale it up as needed.
=> Map2 would be used when the Redfish client subscribe for the ResourceType to get the Dbus mappings.
=> Map3 would be used when the Dbus signal gets generated and need the Redfish mappings.
=> Once we have all thsese mapping gets generated and loaded into the memory, bmcweb would start listening for the interfaces listed in map2.
=> Once any Dbus signal gets generated map3 would be used to get the Redfish mapping.
Please let me know if you have any concerns with this approach.
Ratan
Quoted 38 lines · Jason M
Quoted 7 lines · Ratan Gupta
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
I think I'm still a little confused at the scope. My understanding was that this initial design for EventService was only for monitoring event messages and not resources in general. It seems like it may not make sense to try to use the same tools and approach for both message monitoring and resource monitoring? Do we need to treat them separately for now to simplify the discussion?
Jason, When you say event messages? What do you mean, Do you mean to say "/redfish/v1/Systems/system/Logservices/eventlog"? >
If yes then this should also go as Resource Event, When ever any log entry gets created under System Log (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would notify to the Redfish client saying that "ResourceCreated" with the URL of the Resource.Yes, new entries under "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you could register for specific MessageIDs, so it's not just a generic "new resource" event like others would be.
Can we register for MessageID? I thought client can register for whole registry not a specific Message ID.
I don't really know. I thought that's what the current implementation allowed, but I don't know for sure if it can or should.
Quoted 10 lines
Quoted 7 lines
Quoted 6 lines
After receiving this event Redfish client will do a GET request on the URL(retrieved as part of event) to get the content of the log.
This will become generic infra for all types of events.
What I'm saying is I don't know if there is a good generic solution to cover both the EventLog and all other resources. I believe the current EventService implementation was designed only for EventLog and may not work well for generic resource events.
Can you get me the example payload for EventLog which is going to be sent with the current design? I am not sure how the eventlog and other resources are different.
This is based on the assumption that for a LogService, you can register for a MessageId. If this is not possible, then they might be treated the same.
Quoted 2 lines
For eventLogs also we have the associated D-bus
objects(/xyz/openbmc_project/logging,/xyz/openbmc_project/dump etc)For Intel platforms, we don't use /xyz/openbmc_project/logging, so we don't have D-Bus objects associated with each EventLog LogEntry. We use rsyslog to create a file that contains many LogEntries.
However, as an unrelated side-thought: linking logging to /xyz/openbmc_project/dump made me wonder if there is a possible solution to the logging issue if we treat /xyz/openbmc_project/logging like /xyz/openbmc_project/dump and place a pointer to the log in the D-Bus object instead of the log itself?
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
I would be coming up with few design approaches and downside with each approach to take it to conclusion.
Thanks! What I'm proposing is that we clarify or possibly separate the discussions about EventLog vs. generic resources to avoid confusion and come up with the right solutions for each.
Quoted 3 lines
Ratan
Quoted 5 lines · Ratan Gupta
Hi James,
As you asked for the examples in the thread(https://lists.ozlabs.org/pipermail/openbmc/2020-June/022125.html), I have created the mapping at the following location.
https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2
Thanks for the examples.
Quoted 2 lines
1) Map1: Mapping between redfish resources to Dbus resources
How does this work when the mapping isn't 1:1 vs D-Bus? Most of the time the enums to not match the d-bus enum, or take multiple d-bus interface to distinguish what the value should be. Also how does this work for discovered things, like multiple chassis?
Quoted 17 lines
2) Map2: Mapping between redfish resource types to the ineterested Dbus interfaces
3) Map3: Mapping between Dbus resources to redfish resourcesI tried to cover the following scenario in the above mapping.
* Redfish resource is mapped to multiple Dbus Resources
* Redfish Property is mapped to single Dbus property
* Redfish Property(complex property) is mapped to multiple dbus property.
* Same type of Redfish Resources are mapped to different Dbus Resources
* Redfish node url having multiple regex : Yet to take a look on this.Quoted 1 line
Flow would be like as below
=> In bmcweb each Redfish node represents to a Redfish Resource.
=> Each node will be having it's own mapping between Redfish properties and the Dbus Resources.
Quoted 7 lines
=> Some code on bmcweb will walkthrough on each node during startup , get this mapping from each node and generate
two mappings
1) Reverse mapping (Dbus Resource to Redfish Resource)(MAP3) and
2) mapping between Resource Types to the interested Dbus
interfaces(MAP2)=> To start with we will support few resource types and then scale it up as needed.
I think we need an idea of what the final solution will look like for more complicated properties, or we'll be creating something that isn't future proof.
Quoted 4 lines
=> Map2 would be used when the Redfish client subscribe for the ResourceType to get the Dbus mappings.
=> Map3 would be used when the Dbus signal gets generated and need the Redfish mappings.
Why can't these be the same mapping? I think having 3 different maps makes this very confusing. I also think this is attempting to generalize the problem too early. If you look at the Redfish code to determine some of the more complicated properties, sometimes it takes quite a bit of logic. That logic also would possibly be nice to reuse. Maybe we can take your idea of a match with a callback to some of the already existing property parsing? Obviously it would need some cleaning up, but I could see something with a map of schemas/properties to function pointers for property parsing.
Quoted 3 lines
=> Once we have all thsese mapping gets generated and loaded into the memory, bmcweb would start listening
for the interfaces listed in map2.
Why aren't these compiled in? I don't see why they need to be loaded, could just be in code. Also, they shouldn't be added as matches until there is a subscriber, or we'll add many unneeded matches.
Quoted 6 lines
=> Once any Dbus signal gets generated map3 would be used to get the Redfish mapping.
Please let me know if you have any concerns with this approach.
Ratan
Hi James,
Quoted 5 lines · James Feist
Quoted 4 lines · Ratan Gupta
Hi James,
As you asked for the examples in the thread(https://lists.ozlabs.org/pipermail/openbmc/2020-June/022125.html), I have created the mapping at the following location.
Quoted 8 lines
Thanks for the examples.
Quoted 2 lines
1) Map1: Mapping between redfish resources to Dbus resources
How does this work when the mapping isn't 1:1 vs D-Bus? Most of the time the enums to not match the d-bus enum, or take multiple d-bus interface to distinguish what the value should be. Also how does this work for discovered things, like multiple chassis?
I have not mentioned in the below use cases, but just look at the url https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2#file-gistfile1-txt-L77
<https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2#file-gistfile1-txt-L77>
There we are covering the resources which are getting added dynamically.
Quoted 19 lines
Quoted 19 lines
2) Map2: Mapping between redfish resource types to the ineterested Dbus interfaces
3) Map3: Mapping between Dbus resources to redfish resourcesI tried to cover the following scenario in the above mapping.
* Redfish resource is mapped to multiple Dbus Resources
* Redfish Property is mapped to single Dbus property
* Redfish Property(complex property) is mapped to multiple dbus property.
* Same type of Redfish Resources are mapped to different Dbus Resources
* Redfish node url having multiple regex : Yet to take a look on this.Quoted 1 line
Flow would be like as below
=> In bmcweb each Redfish node represents to a Redfish Resource.
=> Each node will be having it's own mapping between Redfish properties and the Dbus Resources.
Quoted 8 lines
Quoted 6 lines
=> Some code on bmcweb will walkthrough on each node during startup , get this mapping from each node and generate two mappings
1) Reverse mapping (Dbus Resource to Redfish Resource)(MAP3) and
2) mapping between Resource Types to the interested Dbus interfaces(MAP2)=> To start with we will support few resource types and then scale it up as needed.
I think we need an idea of what the final solution will look like for more complicated properties, or we'll be creating something that isn't future proof.
I thought I covered the complicated one in the examples(gist link above) but just let me know the other redfish resource which I can take it in the examples.
Quoted 7 lines
Quoted 5 lines
=> Map2 would be used when the Redfish client subscribe for the ResourceType to get the Dbus mappings.
=> Map3 would be used when the Dbus signal gets generated and need the Redfish mappings.
Why can't these be the same mapping?
To start monitoring for Dbus : We need Dbus Resource Path,
Once we get any Dbus signal, we need to send redfish event which needs redfish resource path hence we need reverse map
Did I get you correctly?
Quoted 7 lines
I think having 3 different maps makes this very confusing. I also think this is attempting to generalize the problem too early. If you look at the Redfish code to determine some of the more complicated properties, sometimes it takes quite a bit of logic. That logic also would possibly be nice to reuse. Maybe we can take your idea of a match with a callback to some of the already existing property parsing? Obviously it would need some cleaning up, but I could see something with a map of schemas/properties to function pointers for property parsing.
Quoted 3 lines
=> Once we have all thsese mapping gets generated and loaded into the memory, bmcweb would start listening for the interfaces listed in map2.
Why aren't these compiled in? I don't see why they need to be loaded, could just be in code. Also, they shouldn't be added as matches until there is a subscriber, or we'll add many unneeded matches.
They would be compiled in, they would be in memory(big std::map).
Agree monitoring should start only when there is a subscriber.
Quoted 7 lines
Quoted 7 lines
=> Once any Dbus signal gets generated map3 would be used to get the Redfish mapping.
Please let me know if you have any concerns with this approach.
Ratan