opensm: Allow connectivity between switches in up down and fat tree

After a second thought and in order not to break current configuration,
I send this modified patch that does not change connect_roots option but changes its functionality in up-down (I think that in fat-tree it is already implemented)

Modify connect_roots option to allow connectivity between all switches in up-down routing algorithm and in this way be fully IBA compliant

Signed-off-by Eli Dorfman<elid-smomgflXvOZWk0Htik3J/[email protected]>

---
  opensm/man/opensm.8.in         |    2 +-
  opensm/opensm/main.c           |    2 +-
  opensm/opensm/osm_ucast_updn.c |    4 +---
  3 files changed, 3 insertions(+), 5 deletions(-)

opensm/man/opensm.8.in +1 -1

index 9053611..c67126e 100644
@@ -174,7 +174,7 @@ the host comes back online.
  .TP
  \fB\-z\fR, \fB\-\-connect_roots\fR
  This option enforces routing engines (up/down and
-fat-tree) to make connectivity between root switches and in
+fat-tree) to make connectivity between all switches and in
  this way to be fully IBA complaint. In many cases this can
  violate "pure" deadlock free algorithm, so use it carefully.
  .TP

opensm/opensm/main.c +1 -1

index 0093aa7..82ca78f 100644
@@ -187,7 +187,7 @@ static void show_usage(void)
  	       "          Sets the SL to use to communicate with the SM/SA. Defaults to 0.\n\n");
  	printf("--connect_roots, -z\n"
  	       "          This option enforces routing engines (up/down and \n"
-	       "          fat-tree) to make connectivity between root switches\n"
+	       "          fat-tree) to make connectivity between all switches\n"
  	       "          and in this way be IBA compliant. In many cases,\n"
  	       "          this can violate \"pure\" deadlock free algorithm, so\n"
  	       "          use it carefully.\n\n");

opensm/opensm/osm_ucast_updn.c +0 -0

index 164c6f4..f44ca24 100644

Quoted 65 lines · Yevgeny Kliteynik

Eli,

Quoted 62 lines · Eli Dorfman (Voltaire)

After a second thought and in order not to break current configuration, I send this modified patch that does not change connect_roots option but changes its functionality in up-down (I think that in fat-tree it is already implemented)

Modify connect_roots option to allow connectivity between all switches in up-down routing algorithm and in this way be fully IBA compliant

Signed-off-by Eli Dorfman<elid-smomgflXvOZWk0Htik3J/[email protected]>

---
   opensm/man/opensm.8.in         |    2 +-
   opensm/opensm/main.c           |    2 +-
   opensm/opensm/osm_ucast_updn.c |    4 +---
   3 files changed, 3 insertions(+), 5 deletions(-)

opensm/man/opensm.8.in +1 -1

index 9053611..c67126e 100644
@@ -174,7 +174,7 @@ the host comes back online.
   .TP
   \fB\-z\fR, \fB\-\-connect_roots\fR
   This option enforces routing engines (up/down and
-fat-tree) to make connectivity between root switches and in
+fat-tree) to make connectivity between all switches and in
   this way to be fully IBA complaint. In many cases this can
   violate "pure" deadlock free algorithm, so use it carefully.
   .TP

opensm/opensm/main.c +0 -0

index 0093aa7..82ca78f 100644
@@ -187,7 +187,7 @@ static void show_usage(void)
              "          Sets the SL to use to communicate with the
 SM/SA. Defaults to 0.\n\n");
       printf("--connect_roots, -z\n"
              "          This option enforces routing engines (up/down

and \n"
- " fat-tree) to make connectivity between root
switches\n"
+ " fat-tree) to make connectivity between all
switches\n"
" and in this way be IBA compliant. In many
cases,\n"
" this can violate \"pure\" deadlock free
algorithm, so\n"

              "          use it carefully.\n\n");

opensm/opensm/osm_ucast_updn.c +1 -3

index 164c6f4..f44ca24 100644
@@ -314,9 +314,7 @@ static int updn_set_min_hop_table(IN updn_t * p_updn)
            item = cl_qmap_next(item)) {
           p_sw = (osm_switch_t *)item;
           /* Clear Min Hop Table */
-        if (p_subn->opt.connect_roots)
-            updn_clear_non_root_hops(p_updn, p_sw);
-        else
+        if (!p_subn->opt.connect_roots)
               osm_switch_clear_hops(p_sw);

Quoted 8 lines

opensm/opensm/osm_ucast_updn.c static int updn_set_min_hop_table(IN updn_t * p_updn) · line 320

Quoted 2 lines

opensm/opensm/osm_ucast_updn.c +0 -0

       }
 

What kind of testing was done for this?
I have a strong feeling that it will break up/down. If the connect_roots option is on, you will not clear the lid matrix at all, and it will contain also the down/up routes.

that is the idea.
instead of leaving only up/dpwn routes we want to keep routes between all switches. the routes between host nodes will still be up/down.

Eli

Quoted 2 lines

-- Yevgeny

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Eli,

Quoted 62 lines · Eli Dorfman (Voltaire)

After a second thought and in order not to break current configuration, I send this modified patch that does not change connect_roots option but changes its functionality in up-down (I think that in fat-tree it is already implemented)

Modify connect_roots option to allow connectivity between all switches in up-down routing algorithm and in this way be fully IBA compliant

Signed-off-by Eli Dorfman<elid-smomgflXvOZWk0Htik3J/[email protected]>

---
  opensm/man/opensm.8.in         |    2 +-
  opensm/opensm/main.c           |    2 +-
  opensm/opensm/osm_ucast_updn.c |    4 +---
  3 files changed, 3 insertions(+), 5 deletions(-)

opensm/man/opensm.8.in +1 -1

index 9053611..c67126e 100644
@@ -174,7 +174,7 @@ the host comes back online.
  .TP
  \fB\-z\fR, \fB\-\-connect_roots\fR
  This option enforces routing engines (up/down and
-fat-tree) to make connectivity between root switches and in
+fat-tree) to make connectivity between all switches and in
  this way to be fully IBA complaint. In many cases this can
  violate "pure" deadlock free algorithm, so use it carefully.
  .TP

opensm/opensm/main.c +0 -0

index 0093aa7..82ca78f 100644
@@ -187,7 +187,7 @@ static void show_usage(void)
             "          Sets the SL to use to communicate with the
 SM/SA. Defaults to 0.\n\n");
      printf("--connect_roots, -z\n"
             "          This option enforces routing engines (up/down

and \n"
- " fat-tree) to make connectivity between root
switches\n"
+ " fat-tree) to make connectivity between all
switches\n"
" and in this way be IBA compliant. In many
cases,\n"
" this can violate \"pure\" deadlock free
algorithm, so\n"

             "          use it carefully.\n\n");

opensm/opensm/osm_ucast_updn.c +1 -3

index 164c6f4..f44ca24 100644
@@ -314,9 +314,7 @@ static int updn_set_min_hop_table(IN updn_t * p_updn)
           item = cl_qmap_next(item)) {
          p_sw = (osm_switch_t *)item;
          /* Clear Min Hop Table */
-        if (p_subn->opt.connect_roots)
-            updn_clear_non_root_hops(p_updn, p_sw);
-        else
+        if (!p_subn->opt.connect_roots)
              osm_switch_clear_hops(p_sw);

opensm/opensm/osm_ucast_updn.c static int updn_set_min_hop_table(IN updn_t * p_updn) · line 320

Quoted 2 lines

opensm/opensm/osm_ucast_updn.c +0 -0

      }
 

What kind of testing was done for this?
I have a strong feeling that it will break up/down. If the connect_roots option is on, you will not clear the lid matrix at all, and it will contain also the down/up routes.

-- Yevgeny

Quoted 77 lines · Eli Dorfman (Voltaire)

Quoted 67 lines · Yevgeny Kliteynik

Eli,

Quoted 64 lines · Eli Dorfman (Voltaire)

After a second thought and in order not to break current configuration, I send this modified patch that does not change connect_roots option but
changes its functionality in up-down (I think that in fat-tree it is already implemented)

Modify connect_roots option to allow connectivity between all switches in up-down routing algorithm and in this way be fully IBA compliant

Signed-off-by Eli Dorfman<elid-smomgflXvOZWk0Htik3J/[email protected]>

---
   opensm/man/opensm.8.in         |    2 +-
   opensm/opensm/main.c           |    2 +-
   opensm/opensm/osm_ucast_updn.c |    4 +---
   3 files changed, 3 insertions(+), 5 deletions(-)

opensm/man/opensm.8.in +1 -1

index 9053611..c67126e 100644
@@ -174,7 +174,7 @@ the host comes back online.
   .TP
   \fB\-z\fR, \fB\-\-connect_roots\fR
   This option enforces routing engines (up/down and
-fat-tree) to make connectivity between root switches and in
+fat-tree) to make connectivity between all switches and in
   this way to be fully IBA complaint. In many cases this can
   violate "pure" deadlock free algorithm, so use it carefully.
   .TP

opensm/opensm/main.c +0 -0

index 0093aa7..82ca78f 100644
@@ -187,7 +187,7 @@ static void show_usage(void)
              "          Sets the SL to use to communicate with the
 SM/SA. Defaults to 0.\n\n");
       printf("--connect_roots, -z\n"
              "          This option enforces routing engines (up/down

and \n"
- " fat-tree) to make connectivity between root
switches\n"
+ " fat-tree) to make connectivity between all
switches\n"
" and in this way be IBA compliant. In many
cases,\n"
" this can violate \"pure\" deadlock free
algorithm, so\n"

              "          use it carefully.\n\n");

opensm/opensm/osm_ucast_updn.c +1 -3

index 164c6f4..f44ca24 100644
@@ -314,9 +314,7 @@ static int updn_set_min_hop_table(IN updn_t *
 p_updn)
            item = cl_qmap_next(item)) {
           p_sw = (osm_switch_t *)item;
           /* Clear Min Hop Table */
-        if (p_subn->opt.connect_roots)
-            updn_clear_non_root_hops(p_updn, p_sw);
-        else
+        if (!p_subn->opt.connect_roots)
               osm_switch_clear_hops(p_sw);

Quoted 8 lines

opensm/opensm/osm_ucast_updn.c static int updn_set_min_hop_table(IN updn_t * · line 320

Quoted 2 lines

opensm/opensm/osm_ucast_updn.c +0 -0

       }

What kind of testing was done for this?
I have a strong feeling that it will break up/down. If the connect_roots option is on, you will not clear the lid matrix at all, and it will contain also the down/up routes.

Quoted 4 lines

that is the idea.
instead of leaving only up/dpwn routes we want to keep routes between all switches.
the routes between host nodes will still be up/down.

Lid matrix might contain down/up routes between many kinds of switches, not only between roots. This means that you might have down/up paths between leaf switches, even though you could have found up/down path for them. This also means that you might end up with down/up routes between HCAs that are connected to these leafs.

-- Yevgeny

Quoted 10 lines

Eli

Quoted 2 lines

-- Yevgeny

Signature · 3 lines

To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html