hadoop
PATCH 1/1 hadoop
Sorry about the spam today. This should work.
Added policy for the hadoop stack to refpolicy. All major components of hadoop have been separated and confined (namenode, datanode, jobtracker, tasktracker, secondarynamenode, zookeeper). Since many of the domains use the same executable to transfer into their domain, the init scripts were labelled with a custom initrc domain.
Quoted 1 line
From there a domain transfer can occur using the same executable. Since the domains share the same directory for
logging and data files, type transfers were only done on files not directories. JMX and the rest of hadoop can continue to run without a problem. The policy was tested against Cloudera's version of hadoop CDH2 and CDH3. An unconfined role transition was also needed to get hadoop in the correct domain. Not sure if we want to add the zookeeper and namenode ports to refpolicy or use semanage. I added them to refpolicy.
--- policy/modules/apps/hadoop.fc | 10 policy/modules/apps/hadoop.if | 211 ++++++++++++++++++++ policy/modules/apps/hadoop.te | 91 ++++++++ policy/modules/kernel/corenetwork.te.in | 4 policy/modules/services/hadoop_datanode.fc | 5 policy/modules/services/hadoop_datanode.if | 19 + policy/modules/services/hadoop_datanode.te | 118 +++++++++++ policy/modules/services/hadoop_jobtracker.fc | 5 policy/modules/services/hadoop_jobtracker.if | 18 + policy/modules/services/hadoop_jobtracker.te | 119 +++++++++++ policy/modules/services/hadoop_namenode.fc | 6 policy/modules/services/hadoop_namenode.if | 18 + policy/modules/services/hadoop_namenode.te | 117 +++++++++++ policy/modules/services/hadoop_secondarynamenode.fc | 6 policy/modules/services/hadoop_secondarynamenode.if | 19 + policy/modules/services/hadoop_secondarynamenode.te | 117 +++++++++++ policy/modules/services/hadoop_tasktracker.fc | 6 policy/modules/services/hadoop_tasktracker.if | 18 + policy/modules/services/hadoop_tasktracker.te | 120 +++++++++++ policy/modules/services/hadoop_zookeeper.fc | 11 + policy/modules/services/hadoop_zookeeper.if | 18 + policy/modules/services/hadoop_zookeeper.te | 115 ++++++++++ policy/modules/system/unconfined.if | 25 ++ 23 files changed, 1196 insertions(+)
policy/modules/system/unconfined.if +25 -0
@@ -279,6 +279,31 @@ interface(`unconfined_domtrans_to',` ######################################## ## <summary> +## Allow a program to enter the specified domain through the +## unconfined role. +## </summary> +## <desc> +## <p> +## Allow unconfined role to execute the specified program in +## the specified domain. +## </p> +## </desc> +## <param name="domain"> +## <summary> +## Domain to execute in. +## </summary> +## </param> +# +interface(`unconfined_roletrans',` + gen_require(` + role unconfined_r; + ') + + role unconfined_r types $1; +') + +######################################## +## <summary> ## Allow unconfined to execute the specified program in ## the specified domain. Allow the specified domain the ## unconfined role and use of unconfined user terminals.
policy/modules/kernel/corenetwork.te.in +1 -0
@@ -105,6 +105,7 @@ network_port(giftd, tcp,1213,s0) network_port(git, tcp,9418,s0, udp,9418,s0) network_port(gopher, tcp,70,s0, udp,70,s0) network_port(gpsd, tcp,2947,s0) +network_port(hadoop_namenode, tcp, 8020,s0) network_port(hddtemp, tcp,7634,s0) network_port(howl, tcp,5335,s0, udp,5353,s0) network_port(hplip, tcp,1782,s0, tcp,2207,s0, tcp,2208,s0, tcp, 8290,s0, tcp,50000,s0, tcp,50002,s0, tcp,8292,s0, tcp,9100,s0, tcp,9101,s0, tcp,9102,s0, tcp,9220,s0, tcp,9221,s0, tcp,9222,s0, tcp,9280,s0, tcp,9281,s0, tcp,9282,s0, tcp,9290,s0, tcp,9291,s0, tcp,9292,s0)
policy/modules/services/hadoop_namenode.fc new file +6 -0
@@ -0,0 +1,6 @@ +/etc/rc\.d/init\.d/hadoop-(.*)?-namenode -- gen_context(system_u:object_r:hadoop_namenode_initrc_exec_t, s0) + +/var/log/hadoop(.*)?/hadoop-hadoop-namenode-(.*)? gen_context(system_u:object_r:hadoop_namenode_log_t, s0) + +/var/lib/hadoop(.*)?/cache/hadoop/dfs/name(/.*)? gen_context(system_u:object_r:hadoop_namenode_data_t, s0) +
policy/modules/services/hadoop_namenode.if new file +18 -0
@@ -0,0 +1,18 @@ +## <summary>Hadoop Namenode Policy</summary> +######################################## +## <summary> +## Give permission to a domain to signull hadoop_namenode_t +## </summary> +## <param name="domain"> +## <summary> +## Domain needing permission +## </summary> +## </param> +# +interface(`hadoop_namenode_signull', ` + gen_require(` + type hadoop_namenode_t; + ') + + allow $1 hadoop_namenode_t:process { signull }; +')
policy/modules/services/hadoop_namenode.te new file +117 -0
117 lines not on this page · raw message
policy/modules/apps/hadoop.fc new file +10 -0
@@ -0,0 +1,10 @@ +/usr/bin/hadoop(.*)? -- gen_context(system_u:object_r:hadoop_exec_t,s0) + +/etc/hadoop(/.*)? gen_context(system_u:object_r:hadoop_etc_t,s0) +/etc/hadoop-0.20(/.*)? gen_context(system_u:object_r:hadoop_etc_t,s0) + +/var/lib/hadoop(.*)? gen_context(system_u:object_r:hadoop_data_t,s0) + +/var/log/hadoop(.*)? gen_context(system_u:object_r:hadoop_log_t,s0) + +/var/run/hadoop(.*)? gen_context(system_u:object_r:hadoop_run_t,s0)
policy/modules/apps/hadoop.if new file +211 -0
211 lines not on this page · raw message
policy/modules/apps/hadoop.te new file +91 -0
91 lines not on this page · raw message
policy/modules/services/hadoop_datanode.fc new file +5 -0
@@ -0,0 +1,5 @@ +/etc/rc\.d/init\.d/hadoop-(.*)?-datanode -- gen_context(system_u:object_r:hadoop_datanode_initrc_exec_t, s0) + +/var/log/hadoop(.*)?/hadoop-hadoop-datanode-(.*)? gen_context(system_u:object_r:hadoop_datanode_log_t, s0) + +/var/lib/hadoop(.*)?/cache/hadoop/dfs/data(/.*)? gen_context(system_u:object_r:hadoop_datanode_data_t, s0)
policy/modules/services/hadoop_datanode.if new file +19 -0
@@ -0,0 +1,19 @@ +## <summary>Hadoop DataNode</summary> + +######################################## +## <summary> +## Give permission to a domain to signull hadoop_datanode_t +## </summary> +## <param name="domain"> +## <summary> +## Domain needing permission +## </summary> +## </param> +# +interface(`hadoop_datanode_signull', ` + gen_require(` + type hadoop_datanode_t; + ') + + allow $1 hadoop_datanode_t:process { signull }; +')
policy/modules/services/hadoop_datanode.te new file +118 -0
118 lines not on this page · raw message
policy/modules/services/hadoop_jobtracker.fc new file +5 -0
@@ -0,0 +1,5 @@ +/etc/rc\.d/init\.d/hadoop-(.*)?-jobtracker -- gen_context(system_u:object_r:hadoop_jobtracker_initrc_exec_t, s0) + +/var/log/hadoop(.*)?/hadoop-hadoop-jobtracker-(.*)? gen_context(system_u:object_r:hadoop_jobtracker_log_t, s0) + +/var/lib/hadoop(.*)?/cache/hadoop/mapred/local/jobTracker(/.*)? gen_context(system_u:object_r:hadoop_jobtracker_data_t, s0)
policy/modules/services/hadoop_jobtracker.if new file +18 -0
@@ -0,0 +1,18 @@ +## <summary>Hadoop JobTracker</summary> +######################################## +## <summary> +## Give permission to a domain to signull hadoop_jobtracker_t +## </summary> +## <param name="domain"> +## <summary> +## Domain needing permission +## </summary> +## </param> +# +interface(`hadoop_jobtracker_signull', ` + gen_require(` + type hadoop_jobtracker_t; + ') + + allow $1 hadoop_jobtracker_t:process { signull }; +')
policy/modules/services/hadoop_jobtracker.te new file +119 -0
119 lines not on this page · raw message
policy/modules/services/hadoop_secondarynamenode.fc new file +6 -0
@@ -0,0 +1,6 @@ +/etc/rc\.d/init\.d/hadoop-(.*)?-secondarynamenode -- gen_context(system_u:object_r:hadoop_secondarynamenode_initrc_exec_t, s0) + +/var/log/hadoop(.*)?/hadoop-hadoop-secondarynamenode-(.*)? gen_context(system_u:object_r:hadoop_secondarynamenode_log_t, s0) + +/var/lib/hadoop(.*)?/cache/hadoop/dfs/namesecondary(/.*)? gen_context(system_u:object_r:hadoop_secondarynamenode_data_t, s0) +
policy/modules/services/hadoop_secondarynamenode.if new file +19 -0
@@ -0,0 +1,19 @@ +## <summary>Hadoop Secondary Namenode</summary> + +######################################## +## <summary> +## Give permission to a domain to signull hadoop_secondarynamenode_t +## </summary> +## <param name="domain"> +## <summary> +## Domain needing permission +## </summary> +## </param> +# +interface(`hadoop_secondarynamenode_signull', ` + gen_require(` + type hadoop_secondarynamenode_t; + ') + + allow $1 hadoop_secondarynamenode_t:process { signull }; +')
policy/modules/services/hadoop_secondarynamenode.te new file +117 -0
117 lines not on this page · raw message
policy/modules/services/hadoop_tasktracker.fc new file +6 -0
@@ -0,0 +1,6 @@ +/etc/rc\.d/init\.d/hadoop-(.*)?-tasktracker -- gen_context(system_u:object_r:hadoop_tasktracker_initrc_exec_t, s0) + +/var/log/hadoop(.*)?/hadoop-hadoop-tasktracker-(.*)? gen_context(system_u:object_r:hadoop_tasktracker_log_t, s0) + +/var/lib/hadoop(.*)?/cache/hadoop/mapred/local/taskTracker(/.*)? gen_context(system_u:object_r:hadoop_tasktracker_data_t, s0) +
policy/modules/services/hadoop_tasktracker.if new file +18 -0
@@ -0,0 +1,18 @@ +## <summary>Hadoop TaskTracker</summary> +######################################## +## <summary> +## Give permission to a domain to signull hadoop_tasktracker_t +## </summary> +## <param name="domain"> +## <summary> +## Domain needing permission +## </summary> +## </param> +# +interface(`hadoop_tasktracker_signull', ` + gen_require(` + type hadoop_tasktracker_t; + ') + + allow $1 hadoop_tasktracker_t:process { signull }; +')
policy/modules/services/hadoop_tasktracker.te new file +120 -0
120 lines not on this page · raw message
policy/modules/kernel/corenetwork.te.in +3 -0
@@ -211,6 +211,9 @@ network_port(whois, tcp,43,s0, udp,43,s0, tcp, 4321, s0 , udp, 4321, s0 ) network_port(xdmcp, udp,177,s0, tcp,177,s0) network_port(xen, tcp,8002,s0) network_port(xfs, tcp,7100,s0) +network_port(zookeeper_client, tcp, 2181,s0) +network_port(zookeeper_election, tcp, 3888,s0) +network_port(zookeeper_leader, tcp, 2888,s0) network_port(xserver, tcp,6000-6020,s0) network_port(zebra, tcp,2600-2604,s0, tcp,2606,s0, udp,2600-2604,s0, udp,2606,s0) network_port(zope, tcp,8021,s0)
policy/modules/services/hadoop_zookeeper.fc new file +11 -0
@@ -0,0 +1,11 @@ +/usr/bin/zookeeper-server -- gen_context(system_u:object_r:zookeeper_server_exec_t, s0) + +/usr/bin/zookeeper-client -- gen_context(system_u:object_r:zookeeper_exec_t, s0) + +/var/log/zookeeper(/.*)? gen_context(system_u:object_r:zookeeper_log_t, s0) + +/var/zookeeper(/.*)? gen_context(system_u:object_r:zookeeper_server_data_t, s0) + +/etc/zookeeper(/.*)? gen_context(system_u:object_r:zookeeper_etc_t, s0) +/etc/zookeeper.dist(/.*)? gen_context(system_u:object_r:zookeeper_etc_t, s0) +
policy/modules/services/hadoop_zookeeper.if new file +18 -0
@@ -0,0 +1,18 @@ +## <summary>Hadoop Zookeeper Server</summary> +######################################## +## <summary> +## Give permission to a domain to signull zookeeper_server_t +## </summary> +## <param name="domain"> +## <summary> +## Domain needing permission +## </summary> +## </param> +# +interface(`zookeeper_server_signull', ` + gen_require(` + type zookeeper_server_t; + ') + + allow $1 zookeeper_server_t:process signull; +')
policy/modules/services/hadoop_zookeeper.te new file +115 -0
115 lines not on this page · raw message
PATCH 1/1 hadoop
Quoted 5 lines · Dominick Grif
Was this policy developed on and for the EL5 system? I am wondering why
unconfined_tis in the mix here. Remember that strict and mls policy do not ship with the unconfined domain, and that in recent refpolicy, the interaction with the unconfined domain should be optional, so that it can be de-installed.
This policy was developed on a Fedora machine. I can see where problems would happen with unconfined_t. I wrapped them with an optional_policy block so it can be run under strict or mls.
Quoted 6 lines
I am wondering why it is
unconfined_tthat is domain transitioning to the rc script domains and not init. I guess the transition fromunconfined_ttoinitrc_tis not happening automatically. In that case i would use therun_initcommand to domain transition to initrc first and then let that domain transition to your rc script domain, probably using theinit_script_domain()interface.
The unconfined_t is domain transitioning to the script domain so admins can restart the daemon. I don't think run_init would help because we don't want to transition into init_t. The policy is a little different because we have one executable for multiple domains. We had to a create pseudo-init domain like hadoop_datanode_initrc_t so init_daemon_domain(hadoop_datanode_initrc_t, hadoop_datanode_initrc_exec_t) could be used.
I changed the patch based on your feedback. Any input is appreciated.
PATCH 1/1 hadoop
Quoted 19 lines · Paul Nuzzi
Quoted 5 lines · Dominick Grif
Was this policy developed on and for the EL5 system? I am wondering why
unconfined_tis in the mix here. Remember that strict and mls policy do not ship with the unconfined domain, and that in recent refpolicy, the interaction with the unconfined domain should be optional, so that it can be de-installed.This policy was developed on a Fedora machine. I can see where problems would happen with
unconfined_t. I wrapped them with anoptional_policyblock so it can be run under strict or mls.Quoted 6 lines
I am wondering why it is
unconfined_tthat is domain transitioning to the rc script domains and not init. I guess the transition fromunconfined_ttoinitrc_tis not happening automatically. In that case i would use therun_initcommand to domain transition to initrc first and then let that domain transition to your rc script domain, probably using theinit_script_domain()interface.
Quoted 6 lines
The
unconfined_tis domain transitioning to the script domain so admins can restart the daemon. I don't thinkrun_initwould help because we don't want to transition intoinit_t. The policy is a little different because we have one executable for multiple domains. We had to a create pseudo-init domain likehadoop_datanode_initrc_tsoinit_daemon_domain(hadoop_datanode_initrc_t, hadoop_datanode_initrc_exec_t)could be used.
I think i know what you mean but i also believe what you want can be accomplished with for example: init_script_domain(hadoop_datanode_initrc_t) I must admit that i never tried it though.
But anyhow, wrapping the unconfined policy calls into optional block does not solve the problem. Even though the policy may build, it is still not usable since nothing else can transition to the domains if the unconfined domain is gone.
unconfined_t -+-> initrc_t -+-> hadoop_domain1_initrc_t --> hadoop_domain1_t
sysadm_t -/ \-> hadoop_domain2_initrc_t --> hadoop_domain2_t
\-> hadoop_domain3_initrc_t --> hadoop_domain3_tThis issue should be solved (if possible) first in my opinion.
Quoted 5 lines
I changed the patch based on your feedback. Any input is appreciated.
_______________________________________________
refpolicy mailing list
refpolicy at oss.tresys.com
http://oss.tresys.com/mailman/listinfo/refpolicy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://oss.tresys.com/pipermail/refpolicy/attachments/20100910/cfd8a0e8/attachment.bin