opensm: Reduce heap consumption by multicast routing tables (MFTs)
PATCH v4 1/1 opensm: Reduce heap consumption by multicast routing tables (MFTs)
Heap memory consumption by the unicast and multicast routing tables can be reduced.
This patch is analagous to the previous patch doing this for the unicast routing tables (LFTs).
Using valgrind --tool=massif (for heap profiling), there are couple of place ->38.75% (11,206,656B) 0x43267E: osm_switch_new (osm_switch.c:134) ->12.89% (3,728,256B) 0x40F8C9: osm_mcast_tbl_init (osm_mcast_tbl.c:96)
osm_mcast_tbl_init (osm_mcast_tbl.c:96):
p_tbl->p_mask_tbl = malloc(p_tbl->num_entries *
(IB_MCAST_POSITION_MAX +
1) * IB_MCAST_MASK_SIZE / 8);
num_entries above is set based on the switch's MulticastFDBCap (indicated in it's SM class SwitchInfo attribute).
MFTs are only be increased in size and are never reduced in size. If a realloc for MFT fails, it is treated as a fatal error and OpenSM is exited.
---
Changes since v3:
Renamed mft_size to mft_depth and added description in osm_mcast_tbl.h Removed vestigial realloc mask tbl call in osm_dump.c Simplified max_mlid determination in alloc_mfts
Added return value to osm_mcast_tbl_realloc_mask_tbl
Added return value to alloc_mfts
Handle alloc_mfts failure in osm_mcast_mgr_process/process_mgroups Renamed osm_mcast_tbl_realloc_mask_tbl to osm_mcast_tbl_realloc In osm_mcast_tbl_realloc, simplified mft_depth calculation
Changes since v2:
MFT allocation during routing preparation rather than on table access
Changes since v1:
MFT allocation based on actual MLID requests
opensm/include/opensm/osm_mcast_tbl.h +38 -5
@@ -1,6 +1,6 @@/* * Copyright (c) 2004, 2005 Voltaire, Inc. All rights reserved. -* Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. +* Copyright (c) 2002-2009 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. * * This software is available to you under a choice of one of two
@@ -74,6 +74,7 @@ typedef struct osm_mcast_fwdbl {int16_t max_block_in_use;uint16_t num_entries;uint16_t max_mlid_ho; +uint16_t mft_depth;uint16_t (*p_mask_tbl)[][IB_MCAST_POSITION_MAX ]; } osm_mcast_tbl_t;/*
@@ -97,7 +98,11 @@ typedef struct osm_mcast_fwdbl {* max_mlid_ho * Maximum MLID value (host order). * -* pp_mask_tbl +* mft_depth +* Number of MLIDs in the currently allocated multicast port mask +* table. +* +* p_mask_tbl * Pointer to a two dimensional array of port_masks for this switch. * The first dimension is MLID, the second dimension is mask position. * This pointer is null for switches that do not support multicast.
@@ -114,8 +119,8 @@ typedef struct osm_mcast_fwdbl {* * SYNOPSIS */ -ib_api_status_t osm_mcast_tbl_init(IN osm_mcast_tbl_t * p_tbl, - IN uint8_t num_ports, IN uint16_t capacity); +void osm_mcast_tbl_init(IN osm_mcast_tbl_t * p_tbl, IN uint8_t num_ports, + IN uint16_t capacity);/* * PARAMETERS * num_ports
@@ -126,7 +131,7 @@ ib_api_status_t osm_mcast_tbl_init(IN osm_mcast_tbl_t * p_tbl,* by this switch. * * RETURN VALUE -* IB_SUCCESS on success. +* None. * * NOTES *
@@ -158,6 +163,34 @@ void osm_mcast_tbl_delete(IN osm_mcast_tbl_t ** pp_tbl);* SEE ALSO *********/ +/****f* OpenSM: Forwarding Table/osm_mcast_tbl_realloc +* NAME +* osm_mcast_tbl_realloc +* +* DESCRIPTION +* This function reallocates the multicast port mask table if necessary. +* +* SYNOPSIS +*/ +int +osm_mcast_tbl_realloc(IN osm_mcast_tbl_t * p_tbl, IN uintn_t mlid_offset); +/* +* PARAMETERS +* +* p_tbl +* [in] Pointer to the Multicast Forwarding Table object. +* +* mlid_offset +* [in] Offset of MLID being accessed. +* +* RETURN VALUE +* Returns 0 on success and non-zero value otherwise. +* +* NOTES +* +* SEE ALSO +*/ +/****f* OpenSM: Forwarding Table/osm_mcast_tbl_destroy * NAME * osm_mcast_tbl_destroy
opensm/opensm/osm_mcast_mgr.c +42 -0
60 lines not on this page · raw message
opensm/opensm/osm_mcast_tbl.c +49 -24
138 lines not on this page · raw message
opensm/opensm/osm_switch.c +2 -5
@@ -136,9 +136,8 @@ osm_switch_t *osm_switch_new(IN osm_node_t * p_node,memset (p_sw->p_prof,0 ,sizeof (*p_sw->p_prof) * num_ports); -if (osm_mcast_tbl_init (&p_sw->mcast_tbl,osm_node_get_num_physp (p_node), -cl_ntoh16 (p_si->mcast_cap))) - goto err; +osm_mcast_tbl_init (&p_sw->mcast_tbl,osm_node_get_num_physp (p_node), +cl_ntoh16 (p_si->mcast_cap));for (port_num =0 ; port_num < num_ports; port_num++)osm_port_prof_construct (&p_sw->p_prof[port_num]);
@@ -508,7 +507,6 @@ static int alloc_lft(IN osm_switch_t * p_sw, uint16_t lids) p_sw->lft = new_lft; p_sw->lft_size = lft_size; } -return 0 ; }
@@ -549,7 +547,6 @@ int osm_switch_prepare_path_rebuild(IN osm_switch_t * p_sw, IN uint16_t max_lids p_sw->num_hops = max_lids +1 ; } p_sw->max_lid_ho = max_lids; -return 0 ; }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Sasha,
Quoted 34 lines · Hal Rosenstock
Heap memory consumption by the unicast and multicast routing tables can be reduced.
This patch is analagous to the previous patch doing this for the unicast routing tables (LFTs).
Using valgrind --tool=massif (for heap profiling), there are couple of place ->38.75% (11,206,656B)
0x43267E:osm_switch_new(osm_switch.c:134) ->12.89% (3,728,256B)0x40F8C9:osm_mcast_tbl_init(osm_mcast_tbl.c:96)
osm_mcast_tbl_init(osm_mcast_tbl.c:96):
p_tbl->p_mask_tbl = malloc(p_tbl->num_entries * (IB_MCAST_POSITION_MAX+
1) *IB_MCAST_MASK_SIZE/ 8);
num_entriesabove is set based on the switch's MulticastFDBCap (indicated in it's SM class SwitchInfo attribute).MFTs are only be increased in size and are never reduced in size. If a realloc for MFT fails, it is treated as a fatal error and OpenSM is exited.
Signed-off-by Hal Rosenstock <[email protected]>---
Changes since v3:
Renamedmft_sizetomft_depthand added description inosm_mcast_tbl.hRemoved vestigial realloc mask tbl call inosm_dump.cSimplifiedmax_mliddetermination inalloc_mfts
Added return value toosm_mcast_tbl_realloc_mask_tbl
Added return value toalloc_mfts
Handlealloc_mftsfailure in osm_mcast_mgr_process/process_mgroups Renamedosm_mcast_tbl_realloc_mask_tbltoosm_mcast_tbl_reallocInosm_mcast_tbl_realloc, simplifiedmft_depthcalculation
Quoted 6 lines
Changes since v2:
MFT allocation during routing preparation rather than on table accessChanges since v1:
MFT allocation based on actual MLID requests
Please ignore this. I'll resubmit once we get past the mcast tbl change for max LID host order.
-- Hal
<snip...>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html