Skip to contentSkip to conversation

x86, UV: BAU performance and error recovery

PATCH In review Cliff Wickman 2 participants 2 messages May 31, 2010 – Jun 1, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 x86, UV: BAU performance and error recovery

Ingo,
This patch updates the patch of the same name, which is in your lastest -tip
b8f7fb1: x86, UV: Improve BAU performance and error recovery
It does not depend on any other patch.

It includes the BAU tunables interfaced through debugfs (instead of /proc) per Greg Kroah-Hartman's recommendation.

This patch
- corrects several bugs:
o the lack of BAU messaging to blade 0 (an uninitialized field)
o correct message type for a normal shootdown message
o fix to recovery from timed-out s/w acknowledge resources
o calculation of the actual destination timeout so that a 'short' timeout can be identified
o fix mishandling of non-consecutive hub and socket numbers
o identification of a cpu's socket number
o removal of the check for the h/w stay-busy bug, as under extreme pressure it cannot be accurately detected
o fall back to IPI-method when extreme pressure makes numalink very slow, and then resume BAU-method after an interval
- adds BAU broadcasting to the cpu's on the local node
- simplifies the returning of the original cpu bitmap when the BAU attempt is given up entirely
- adds modification of tuning variables through debugfs (/sys/kernel/debug/sgi_uv/bau_tunables)

Peter, I weighed the review-ability of a patch for each bug fix and each addition, versus the ease of tracking a single patch. And decided on the latter since these bugs and features are only of interest to SGI UV. Hope that is reasonable.

Diffed against 2.6.34 -tip (Sheep on Meth)

---
 arch/x86/include/asm/uv/uv_bau.h |  151 ++++++---
 arch/x86/kernel/tlb_uv.c         |  640 +++++++++++++++++++++++++++------------
 2 files changed, 554 insertions(+), 237 deletions(-)
arch/x86/include/asm/uv/uv_bau.h +102 -49
Index: 100528.linux-tip/arch/x86/include/asm/uv/uv_bau.h
===================================================================
@@ -34,6 +34,7 @@
  */
 
 #define UV_ITEMS_PER_DESCRIPTOR		8
+/* the 'throttle' to prevent the hardware stay-busy bug */
 #define MAX_BAU_CONCURRENT		3
 #define UV_CPUS_PER_ACT_STATUS		32
 #define UV_ACT_STATUS_MASK		0x3
@@ -45,10 +46,26 @@
 #define UV_DESC_BASE_PNODE_SHIFT	49
 #define UV_PAYLOADQ_PNODE_SHIFT		49
 #define UV_PTC_BASENAME			"sgi_uv/ptc_statistics"
+#define UV_BAU_BASENAME			"sgi_uv/bau_tunables"
+#define UV_BAU_TUNABLES_DIR		"sgi_uv"
+#define UV_BAU_TUNABLES_FILE		"bau_tunables"
+#define WHITESPACE			" \t\n"
 #define uv_physnodeaddr(x)		((__pa((unsigned long)(x)) & uv_mmask))
 #define UV_ENABLE_INTD_SOFT_ACK_MODE_SHIFT 15
 #define UV_INTD_SOFT_ACK_TIMEOUT_PERIOD_SHIFT 16
-#define UV_INTD_SOFT_ACK_TIMEOUT_PERIOD 0x000000000bUL
+#define UV_INTD_SOFT_ACK_TIMEOUT_PERIOD 0x0000000009UL
+/* [19:16] SOFT_ACK timeout period  19: 1 is urgency 7  17:16 1 is multiplier */
+#define BAU_MISC_CONTROL_MULT_MASK 3
+
+#define UVH_AGING_PRESCALE_SEL 0x000000b000UL
+/* [30:28] URGENCY_7  an index into a table of times */
+#define BAU_URGENCY_7_SHIFT 28
+#define BAU_URGENCY_7_MASK 7
+
+#define UVH_TRANSACTION_TIMEOUT 0x000000b200UL
+/* [45:40] BAU - BAU transaction timeout select - a multiplier */
+#define BAU_TRANS_SHIFT 40
+#define BAU_TRANS_MASK 0x3f
 
 /*
  * bits in UVH_LB_BAU_SB_ACTIVATION_STATUS_0/1
@@ -59,24 +76,21 @@
 #define DESC_STATUS_SOURCE_TIMEOUT	3
 
 /*
- * source side threshholds at which message retries print a warning
+ * delay for 'plugged' timeout retries, in microseconds
  */
-#define SOURCE_TIMEOUT_LIMIT		20
-#define DESTINATION_TIMEOUT_LIMIT	20
-
-/*
- * misc. delays, in microseconds
- */
-#define THROTTLE_DELAY			10
-#define TIMEOUT_DELAY			10
-#define BIOS_TO				1000
-/* BIOS is assumed to set the destination timeout to 1003520 nanoseconds */
+#define PLUGGED_DELAY			10
 
 /*
  * threshholds at which to use IPI to free resources
  */
+/* after this # consecutive 'plugged' timeouts, use IPI to release resources */
 #define PLUGSB4RESET 100
-#define TIMEOUTSB4RESET 100
+/* after this many consecutive timeouts, use IPI to release resources */
+#define TIMEOUTSB4RESET 1
+/* at this number uses of IPI to release resources, giveup the request */
+#define IPI_RESET_LIMIT 1
+/* after this # consecutive successes, bump up the throttle if it was lowered */
+#define COMPLETE_THRESHOLD 5
 
 /*
  * number of entries in the destination side payload queue
@@ -96,6 +110,13 @@
 #define FLUSH_COMPLETE			4
 
 /*
+ * tuning the action when the numalink network is extremely delayed
+ */
+#define CONGESTED_RESPONSE_US 1000 /* 'long' response time, in microseconds */
+#define CONGESTED_REPS 10 /* long delays averaged over this many broadcasts */
+#define CONGESTED_PERIOD 30 /* time for the bau to be disabled, in seconds */
+
+/*
  * Distribution: 32 bytes (256 bits) (bytes 0-0x1f of descriptor)
  * If the 'multilevel' flag in the header portion of the descriptor
  * has been set to 0, then endpoint multi-unicast mode is selected.
@@ -300,37 +321,16 @@ struct bau_payload_queue_entry {
 	/* bytes 24-31 */
 };
 
-/*
- * one per-cpu; to locate the software tables
- */
-struct bau_control {
-	struct bau_desc *descriptor_base;
+struct msg_desc {
+	struct bau_payload_queue_entry *msg;
+	int msg_slot;
+	int sw_ack_slot;
 	struct bau_payload_queue_entry *va_queue_first;
 	struct bau_payload_queue_entry *va_queue_last;
-	struct bau_payload_queue_entry *bau_msg_head;
-	struct bau_control *uvhub_master;
-	struct bau_control *socket_master;
-	unsigned long timeout_interval;
-	atomic_t active_descriptor_count;
-	int max_concurrent;
-	int max_concurrent_constant;
-	int retry_message_scans;
-	int plugged_tries;
-	int timeout_tries;
-	int ipi_attempts;
-	int conseccompletes;
-	short cpu;
-	short uvhub_cpu;
-	short uvhub;
-	short cpus_in_socket;
-	short cpus_in_uvhub;
-	unsigned short message_number;
-	unsigned short uvhub_quiesce;
-	short socket_acknowledge_count[DEST_Q_SIZE];
-	cycles_t send_message;
-	spinlock_t masks_lock;
-	spinlock_t uvhub_lock;
-	spinlock_t queue_lock;
+};
+
+struct reset_args {
+	int sender;
 };
 
 /*
@@ -344,18 +344,25 @@ struct ptc_stats {
 	unsigned long s_dtimeout; /* destination side timeouts */
 	unsigned long s_time; /* time spent in sending side */
 	unsigned long s_retriesok; /* successful retries */
-	unsigned long s_ntargcpu; /* number of cpus targeted */
-	unsigned long s_ntarguvhub; /* number of uvhubs targeted */
-	unsigned long s_ntarguvhub16; /* number of times >= 16 target hubs */
-	unsigned long s_ntarguvhub8; /* number of times >= 8 target hubs */
-	unsigned long s_ntarguvhub4; /* number of times >= 4 target hubs */
-	unsigned long s_ntarguvhub2; /* number of times >= 2 target hubs */
-	unsigned long s_ntarguvhub1; /* number of times == 1 target hub */
+	unsigned long s_ntargcpu; /* total number of cpu's targeted */
+	unsigned long s_ntargself; /* times the sending cpu was targeted */
+	unsigned long s_ntarglocals; /* targets of cpus on the local blade */
+	unsigned long s_ntargremotes; /* targets of cpus on remote blades */
+	unsigned long s_ntarglocaluvhub; /* targets of the local hub */
+	unsigned long s_ntargremoteuvhub; /* remotes hubs targeted */
+	unsigned long s_ntarguvhub; /* total number of uvhubs targeted */
+	unsigned long s_ntarguvhub16; /* number of times target hubs >= 16*/
+	unsigned long s_ntarguvhub8; /* number of times target hubs >= 8 */
+	unsigned long s_ntarguvhub4; /* number of times target hubs >= 4 */
+	unsigned long s_ntarguvhub2; /* number of times target hubs >= 2 */
+	unsigned long s_ntarguvhub1; /* number of times target hubs == 1 */
 	unsigned long s_resets_plug; /* ipi-style resets from plug state */
 	unsigned long s_resets_timeout; /* ipi-style resets from timeouts */
 	unsigned long s_busy; /* status stayed busy past s/w timer */
 	unsigned long s_throttles; /* waits in throttle */
 	unsigned long s_retry_messages; /* retry broadcasts */
+	unsigned long s_bau_reenabled; /* for bau enable/disable */
+	unsigned long s_bau_disabled; /* for bau enable/disable */
 	/* destination statistics */
 	unsigned long d_alltlb; /* times all tlb's on this cpu were flushed */
 	unsigned long d_onetlb; /* times just one tlb on this cpu was flushed */
@@ -370,6 +377,52 @@ struct ptc_stats {
 	unsigned long d_rcanceled; /* number of messages canceled by resets */
 };
 
+/*
+ * one per-cpu; to locate the software tables
+ */
+struct bau_control {
+	struct bau_desc *descriptor_base;
+	struct bau_payload_queue_entry *va_queue_first;
+	struct bau_payload_queue_entry *va_queue_last;
+	struct bau_payload_queue_entry *bau_msg_head;
+	struct bau_control *uvhub_master;
+	struct bau_control *socket_master;
+	struct ptc_stats *statp;
+	unsigned long timeout_interval;
+	unsigned long set_bau_on_time;
+	atomic_t active_descriptor_count;
+	int plugged_tries;
+	int timeout_tries;
+	int ipi_attempts;
+	int conseccompletes;
+	int baudisabled;
+	int set_bau_off;
+	short cpu;
+	short uvhub_cpu;
+	short uvhub;
+	short cpus_in_socket;
+	short cpus_in_uvhub;
+	unsigned short message_number;
+	unsigned short uvhub_quiesce;
+	short socket_acknowledge_count[DEST_Q_SIZE];
+	cycles_t send_message;
+	spinlock_t uvhub_lock;
+	spinlock_t queue_lock;
+	/* tunables */
+	int max_bau_concurrent;
+	int max_bau_concurrent_constant;
+	int plugged_delay;
+	int plugsb4reset;
+	int timeoutsb4reset;
+	int ipi_reset_limit;
+	int complete_threshold;
+	int congested_response_us;
+	int congested_reps;
+	int congested_period;
+	cycles_t period_time;
+	long period_requests;
+};
+
 static inline int bau_uvhub_isset(int uvhub, struct bau_target_uvhubmask *dstp)
 {
 	return constant_test_bit(uvhub, &dstp->bits[0]);
arch/x86/kernel/tlb_uv.c +452 -188

1,020 lines not on this page · raw message

#1 lore raw
Quoted 33 lines · Cliff Wickman

Ingo,
This patch updates the patch of the same name, which is in your lastest -tip
b8f7fb1: x86, UV: Improve BAU performance and error recovery
It does not depend on any other patch.

It includes the BAU tunables interfaced through debugfs (instead of /proc) per Greg Kroah-Hartman's recommendation.

This patch
- corrects several bugs:
o the lack of BAU messaging to blade 0 (an uninitialized field)
o correct message type for a normal shootdown message
o fix to recovery from timed-out s/w acknowledge resources
o calculation of the actual destination timeout so that a 'short' timeout can be identified
o fix mishandling of non-consecutive hub and socket numbers
o identification of a cpu's socket number
o removal of the check for the h/w stay-busy bug, as under extreme pressure it cannot be accurately detected
o fall back to IPI-method when extreme pressure makes numalink very slow, and then resume BAU-method after an interval
- adds BAU broadcasting to the cpu's on the local node
- simplifies the returning of the original cpu bitmap when the BAU attempt is given up entirely
- adds modification of tuning variables through debugfs (/sys/kernel/debug/sgi_uv/bau_tunables)

Peter, I weighed the review-ability of a patch for each bug fix and each addition, versus the ease of tracking a single patch. And decided on the latter since these bugs and features are only of interest to SGI UV. Hope that is reasonable.

Quoted 8 lines

Diffed against 2.6.34 -tip (Sheep on Meth)

---
 arch/x86/include/asm/uv/uv_bau.h |  151 ++++++---
 arch/x86/kernel/tlb_uv.c         |  640 +++++++++++++++++++++++++++------------
 2 files changed, 554 insertions(+), 237 deletions(-)

Such a large patch cannot go into 2.6.35 - we are after -rc1 already.

So if you want the fixes to be in 2.6.35 then please separate them out into small, digestable patches.

As far as the feature work goes, it looks rather ugly:

Quoted 14 lines
arch/x86/kernel/tlb_uv.c +3 -3
+int uv_flush_send_and_wait(struct bau_desc *bau_desc,
+			   struct cpumask *flush_mask, struct bau_control *bcp)
 {
 	int right_shift;
-	int uvhub;
-	int bit;
 	int completion_status = 0;
 	int seq_number = 0;
 	long try = 0;
 	int cpu = bcp->uvhub_cpu;
+	int tcpu;
 	int this_cpu = bcp->cpu;
-	int this_uvhub = bcp->uvhub;
 	unsigned long mmr_offset;

arch/x86/kernel/tlb_uv.c static inline int atomic_inc_unless_ge(s · line 517

Quoted 10 lines
arch/x86/kernel/tlb_uv.c +4 -1
 	unsigned long index;
 	cycles_t time1;
 	cycles_t time2;
-	struct ptc_stats *stat = &per_cpu(ptcstats, bcp->cpu);
+	cycles_t elapsed;
+	cycles_t local_congested_cycles = congested_cycles;
+	struct ptc_stats *stat = bcp->statp;
 	struct bau_control *smaster = bcp->socket_master;
 	struct bau_control *hmaster = bcp->uvhub_master;
+	struct bau_control *tbcp;

There is absolutely no reason for uv_flush_send_and_wait() to be a 150+ lines long monster-function ... Please reduce the size of functions in tlb_uv.c by introducing proper helper functions.

Also, instead of dropping a mega-patch per cycle you should split out the changes some more, so that they are more reviewable. This current patch should have come as about 10 patches. Each patch should do one well-defined thing, not a laundry list of changes integrated up over a few weeks/months.

Thanks,

Ingo