rteval: Add MCP server for result analysis
cover rteval: Add MCP server for result analysis
This patch series adds an MCP (Model Context Protocol) server for querying and analyzing rteval test results. The server enables AI assistants and other MCP clients to programmatically access rteval data for multi-run analysis, regression detection, and performance trend analysis.
The MCP server provides tools to:
- List and parse rteval XML result files
- Compare multiple test runs side-by-side
- Filter results by kernel version, date range, and latency thresholds
- Extract histogram data and calculate latency percentiles
- Analyze per-CPU latency statistics to identify problematic cores
- Access log files with grep/head/tail filtering
- Compare results against baselines and detect regressions
Target users:
- System administrators running rteval for real-time kernel validation
- QE engineers performing performance regression testing
- CI/CD pipelines requiring automated rteval result analysis
- Developers analyzing system behavior under load
Key features:
- Standards-based MCP protocol implementation
- Comprehensive test suite with integration tests
- Support for both cyclictest and timerlat histogram formats
- Handles overflow buckets and varying histogram sizes
- GPL-2.0-or-later licensed
Dependencies:
- python3-mcp (Fedora package available)
- python3-lxml (standard XML parsing)
Testing:
All tests pass via 'make test' in rteval_mcp/ directory. The server has been tested with Claude Code as an MCP client.
Installation:
The server can be run from git checkout (./rteval_mcp/rteval-mcp-server) or installed system-wide. An example MCP configuration file is provided in mcp.json.example for packagers.
John Kacur (23):
rteval: Add prototype MCP server for result analysis
rteval: Implement compare_results functionality in MCP server
rteval: Add log file access tools to MCP server
rteval: mcp-server: Add Claude Code plugin configuration
rteval: Add .mcp.json for project-level MCP server configuration
rteval: mcp-server: Add rtla command line parser
rteval: mcp-server: Add batch analysis tool
rteval: mcp-server: Add query and filter capabilities rteval MCP: Add histogram extraction and percentile analysis rteval MCP: Reorganize tests into tests/ subdirectory rteval MCP: Fix test_new_tools.py to find rteval result files rteval MCP: Fix glob patterns to find rteval result files rteval MCP: Add integration tests and remove hardcoded examples rteval MCP: Add per-CPU latency statistics breakdown
mcp-server: Add overflow bucket support and cyclictest compatibility
mcp-server: Make tests generic and add Makefile integration
mcp-server: Add load information extraction and filtering
mcp-server: Rename server from rteval to rteval-mcp
mcp-server: Add GPL-2.0-or-later license headers to all files
mcp-server: Add Makefile for running tests
mcp-server: Add demo guide with example queries
mcp-server: Restructure as Python package with dynamic paths
mcp-server: Use relative path in .mcp.json and add example for
installation
.mcp.json | 7 + Makefile | 10 +- mcp.json.example | 11 + rteval_mcp/.claude-plugin/plugin.json | 18 + rteval_mcp/DEMO.md | 462 +++++ rteval_mcp/DEPENDENCIES.md | 33 + rteval_mcp/Makefile | 51 + rteval_mcp/README.md | 490 ++++++ rteval_mcp/__init__.py | 13 + rteval_mcp/__main__.py | 33 + rteval_mcp/manifest.json | 18 + rteval_mcp/rteval-mcp-server | 21 + rteval_mcp/rteval_server.py | 2019 ++++++++++++++++++++++ rteval_mcp/tests/test_histogram.py | 82 + rteval_mcp/tests/test_mcp_integration.py | 220 +++ rteval_mcp/tests/test_new_tools.py | 142 ++ rteval_mcp/tests/test_per_cpu_stats.py | 134 ++ 17 files changed, 3763 insertions(+), 1 deletion(-) create mode 100644 .mcp.json create mode 100644 mcp.json.example create mode 100644 rteval_mcp/.claude-plugin/plugin.json create mode 100644 rteval_mcp/DEMO.md create mode 100644 rteval_mcp/DEPENDENCIES.md create mode 100644 rteval_mcp/Makefile create mode 100644 rteval_mcp/README.md create mode 100644 rteval_mcp/__init__.py create mode 100644 rteval_mcp/__main__.py create mode 100644 rteval_mcp/manifest.json create mode 100755 rteval_mcp/rteval-mcp-server create mode 100755 rteval_mcp/rteval_server.py create mode 100644 rteval_mcp/tests/test_histogram.py create mode 100644 rteval_mcp/tests/test_mcp_integration.py create mode 100644 rteval_mcp/tests/test_new_tools.py create mode 100644 rteval_mcp/tests/test_per_cpu_stats.py
Signature · 1 line
2.55.0
PATCH 1/23 rteval: Add prototype MCP server for result analysis
Add a Model Context Protocol (MCP) server to provide tools for querying and analyzing rteval test results. This is a first prototype focused on multi-run analysis rather than single-result querying.
The MCP server provides three initial tools:
- list_results: Discover rteval XML files in a directory
- parse_result: Extract key metrics from rteval output
- compare_results: Compare two result files (placeholder)
The parser correctly handles rteval 3.11 XML format including:
- Run information (duration, timestamp)
- System information (OS, kernel, architecture)
- Timerlat and cyclictest measurement statistics
Target users are QE engineers and developers who run rteval regularly for RT kernel validation, performance regression testing, and CI/CD.
Tested with rteval 3.11 output on Fedora 43.
--- mcp-server/DEPENDENCIES.md | 25 ++++ mcp-server/README.md | 40 ++++++ mcp-server/server.py | 256 +++++++++++++++++++++++++++++++++++++ 3 files changed, 321 insertions(+) create mode 100644 mcp-server/DEPENDENCIES.md create mode 100644 mcp-server/README.md create mode 100755 mcp-server/server.py
mcp-server/DEPENDENCIES.md new file +25 -0
@@ -0,0 +1,25 @@ +# Dependencies + +## Fedora Packages + +Install these packages on Fedora: + +```bash +sudo dnf install python3-mcp python3-mcp+cli +``` + +- **python3-mcp** - Base MCP SDK for building the server +- **python3-mcp+cli** - CLI tools for development and testing + +## Python Dependencies + +For parsing rteval results: +- **lxml** - XML parsing (should already be available on Fedora as `python3-lxml`) + +```bash +sudo dnf install python3-lxml +``` + +## Optional + +- **python3-mcp+rich** - For enhanced terminal output (optional)
mcp-server/README.md new file +40 -0
@@ -0,0 +1,40 @@ +# rteval MCP Server + +An MCP (Model Context Protocol) server for querying and analyzing rteval test results. + +## Purpose + +This MCP server provides tools to: +- List and discover rteval result files +- Parse rteval XML/JSON output +- Compare multiple test runs +- Query historical performance data +- Analyze latency trends across runs + +## Target Users + +System administrators, QE engineers, and developers who run rteval regularly for: +- Real-time kernel validation +- Performance regression testing +- CI/CD pipelines +- System certification + +## Current Status + +**Prototype/Development** - This is a first attempt at building an MCP server for rteval. +The initial focus is on multi-run analysis rather than single-result querying. + +## Installation + +```bash +pip install -r requirements.txt +``` + +## Usage + +TBD - Server configuration and client usage instructions to be added. + +## Development + +This is a work in progress. The MCP server is being developed in a branch of the +main rteval repository and may be split into a separate repository later.
mcp-server/server.py new file +256 -0
256 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 2/23 rteval: Implement compare_results functionality in MCP server
In reply to John KacurAdd full comparison logic to the compare_results tool. The tool now:
- Extracts structured data from both rteval XML files
- Compares system information (kernel, OS, architecture)
- Compares run durations
- Compares timerlat/cyclictest statistics with detailed metrics
- Calculates differences and percentage changes
- Handles both timerlat and cyclictest measurement types
Added helper function extract_rteval_data() to parse XML files and extract key information in a structured format for easy comparison.
Tested with two rteval 3.11 runs showing consistent results:
- Maximum latency: 3499 vs 3495 us (-0.11%)
- Mean latency: 4.22 vs 4.22 us (+0.09%)
This enables users to track performance trends and identify regressions across multiple test runs.
--- mcp-server/server.py | 200 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 194 insertions(+), 6 deletions(-)
mcp-server/server.py +194 -6
@@ -21,6 +21,75 @@ from mcp.types import Tool, TextContent app = Server("rteval-mcp-server") +def extract_rteval_data(file_path: str) -> dict[str, Any]: + """Extract key data from an rteval XML file for comparison.""" + tree = ET.parse(file_path) + root = tree.getroot() + + data = { + "file": file_path, + "rteval_version": root.get("version", "unknown"), + "run_info": {}, + "system_info": {}, + "measurements": {} + } + + # Run information + runinfo = root.find("run_info") + if runinfo is not None: + data["run_info"] = { + "days": int(runinfo.get("days", "0")), + "hours": int(runinfo.get("hours", "0")), + "minutes": int(runinfo.get("minutes", "0")), + "seconds": int(runinfo.get("seconds", "0")), + } + date_elem = runinfo.find("date") + time_elem = runinfo.find("time") + if date_elem is not None and time_elem is not None: + data["run_info"]["date"] = date_elem.text + data["run_info"]["time"] = time_elem.text + + # System information + sysinfo = root.find("SystemInfo") + if sysinfo is not None: + uname = sysinfo.find("uname") + if uname is not None: + for elem in ["baseos", "node", "arch", "kernel"]: + node = uname.find(elem) + if node is not None: + if elem == "kernel": + data["system_info"]["kernel"] = node.text + data["system_info"]["is_RT"] = node.get("is_RT", "0") == "1" + else: + data["system_info"][elem] = node.text + + # Timerlat measurements + timerlat = root.find(".//timerlat") + if timerlat is not None: + data["measurements"]["timerlat"] = {} + stats = timerlat.find(".//statistics") + if stats is not None: + for stat in stats: + data["measurements"]["timerlat"][stat.tag] = { + "value": stat.text, + "unit": stat.get("unit", "") + } + + # Cyclictest measurements + cyclictest = root.find(".//cyclictest") + if cyclictest is not None: + data["measurements"]["cyclictest"] = {} + stats = cyclictest.find(".//statistics") + if stats is not None: + for stat in stats: + data["measurements"]["cyclictest"][stat.tag] = { + "value": stat.text, + "unit": stat.get("unit", "") + } + + return data + + @app.list_tools() async def list_tools() -> list[Tool]: """List available tools for rteval analysis."""
@@ -227,12 +296,131 @@ async def call_tool(name: str, arguments: Any) -> list[TextContent]: file1 = arguments["file1"] file2 = arguments["file2"] - # For now, just return a placeholder - # This would need actual comparison logic based on rteval structure - return [TextContent( - type="text", - text=f"Comparison tool not yet implemented.\nWould compare:\n {file1}\n {file2}\n" - )] + try: + # Check if files exist + if not Path(file1).exists(): + return [TextContent( + type="text", + text=f"Error: File '{file1}' does not exist" + )] + if not Path(file2).exists(): + return [TextContent( + type="text", + text=f"Error: File '{file2}' does not exist" + )] + + # Extract data from both files + data1 = extract_rteval_data(file1) + data2 = extract_rteval_data(file2) + + # Build comparison report + result = "rteval Results Comparison\n" + result += "=" * 60 + "\n\n" + + result += f"File 1: {Path(file1).name}\n" + result += f"File 2: {Path(file2).name}\n\n" + + # Compare system information + result += "System Comparison:\n" + result += "-" * 60 + "\n" + + sys1 = data1.get("system_info", {}) + sys2 = data2.get("system_info", {}) + + for key in ["baseos", "node", "arch", "kernel", "is_RT"]: + val1 = sys1.get(key, "N/A") + val2 = sys2.get(key, "N/A") + if val1 != val2: + result += f" {key}:\n" + result += f" File 1: {val1}\n" + result += f" File 2: {val2}\n" + else: + result += f" {key}: {val1}\n" + result += "\n" + + # Compare run durations + result += "Run Duration:\n" + result += "-" * 60 + "\n" + + run1 = data1.get("run_info", {}) + run2 = data2.get("run_info", {}) + + def format_duration(info): + return f"{info.get('days', 0)}d {info.get('hours', 0)}h {info.get('minutes', 0)}m {info.get('seconds', 0)}s" + + result += f" File 1: {format_duration(run1)}\n" + result += f" File 2: {format_duration(run2)}\n\n" + + # Compare measurements (timerlat or cyclictest) + meas1 = data1.get("measurements", {}) + meas2 = data2.get("measurements", {}) + + # Determine which measurement type to compare + measurement_type = None + if "timerlat" in meas1 and "timerlat" in meas2: + measurement_type = "timerlat" + elif "cyclictest" in meas1 and "cyclictest" in meas2: + measurement_type = "cyclictest" + + if measurement_type: + result += f"{measurement_type.capitalize()} Statistics Comparison:\n" + result += "-" * 60 + "\n" + + stats1 = meas1.get(measurement_type, {}) + stats2 = meas2.get(measurement_type, {}) + + # Key metrics to compare + for metric in ["samples", "minimum", "maximum", "mean", "median", "standard_deviation"]: + if metric in stats1 and metric in stats2: + val1_data = stats1[metric] + val2_data = stats2[metric] + + val1 = val1_data["value"] + val2 = val2_data["value"] + unit = val1_data.get("unit", "") + + result += f" {metric}:\n" + result += f" File 1: {val1} {unit}\n" + result += f" File 2: {val2} {unit}\n" + + # Calculate difference for numeric values + try: + num1 = float(val1) + num2 = float(val2) + diff = num2 - num1 + if diff > 0: + result += f" Difference: +{diff:.2f} {unit} (File 2 is higher)\n" + elif diff < 0: + result += f" Difference: {diff:.2f} {unit} (File 2 is lower)\n" + else: + result += f" Difference: No change\n" + + # Calculate percentage change for non-zero values + if num1 != 0 and metric != "samples": + pct_change = ((num2 - num1) / num1) * 100 + result += f" Percent change: {pct_change:+.2f}%\n" + + except (ValueError, TypeError): + pass + + result += "\n" + else: + result += "Measurement Comparison: Different measurement types used\n" + result += f" File 1: {', '.join(meas1.keys())}\n" + result += f" File 2: {', '.join(meas2.keys())}\n\n" + + return [TextContent(type="text", text=result)] + + except ET.ParseError as e: + return [TextContent( + type="text", + text=f"Error parsing XML file: {str(e)}" + )] + except Exception as e: + return [TextContent( + type="text", + text=f"Error comparing results: {str(e)}" + )] else: return [TextContent(
Signature · 1 line
2.55.0
PATCH 3/23 rteval: Add log file access tools to MCP server
In reply to John KacurAdd two new tools for accessing rteval log files:
1. list_logs - Lists all log files in an rteval result directory
- Groups by stdout and stderr
- Shows file sizes and paths
- Helps discover what logs are available
2. read_log - Reads content from log files with filtering options
- head: Read first N lines
- tail: Read last N lines
- grep: Search for pattern in log
- Default: Shows first 100 lines
- Handles large files gracefully (timerlat.stdout can be 1.5MB+)
These tools are useful for debugging issues like the timerlat warnings we encountered, allowing users to examine load and measurement logs without manual file access.
Tested with rteval-20260605-2 logs directory containing 8 log files (hackbench, kcompile, timerlat stdout/stderr).
--- mcp-server/server.py | 157 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 157 insertions(+)
mcp-server/server.py +157 -0
@@ -145,6 +145,46 @@ async def list_tools() -> list[Tool]: "required": ["file1", "file2"], }, ), + Tool( + name="list_logs", + description="List available log files in an rteval result directory", + inputSchema={ + "type": "object", + "properties": { + "result_dir": { + "type": "string", + "description": "Path to rteval result directory (e.g., rteval-20260605-1)", + }, + }, + "required": ["result_dir"], + }, + ), + Tool( + name="read_log", + description="Read content from an rteval log file", + inputSchema={ + "type": "object", + "properties": { + "log_path": { + "type": "string", + "description": "Path to the log file", + }, + "tail": { + "type": "integer", + "description": "Read last N lines (optional)", + }, + "head": { + "type": "integer", + "description": "Read first N lines (optional)", + }, + "grep": { + "type": "string", + "description": "Search for pattern in log (optional)", + }, + }, + "required": ["log_path"], + }, + ), ]
@@ -422,6 +462,123 @@ async def call_tool(name: str, arguments: Any) -> list[TextContent]: text=f"Error comparing results: {str(e)}" )] + elif name == "list_logs": + result_dir = arguments["result_dir"] + + try: + dir_path = Path(result_dir) + if not dir_path.exists(): + return [TextContent( + type="text", + text=f"Error: Directory '{result_dir}' does not exist" + )] + + logs_dir = dir_path / "logs" + if not logs_dir.exists(): + return [TextContent( + type="text", + text=f"Error: Logs directory not found in '{result_dir}'" + )] + + log_files = sorted(logs_dir.glob("*")) + + if not log_files: + return [TextContent( + type="text", + text=f"No log files found in '{logs_dir}'" + )] + + result = f"Log files in {logs_dir}:\n\n" + + # Group by type (stdout vs stderr) + stdout_files = [f for f in log_files if f.name.endswith('.stdout')] + stderr_files = [f for f in log_files if f.name.endswith('.stderr')] + + if stdout_files: + result += "Standard Output Logs:\n" + for f in stdout_files: + size = f.stat().st_size + result += f" {f.name}\n" + result += f" Size: {size:,} bytes\n" + result += f" Path: {f}\n" + result += "\n" + + if stderr_files: + result += "Standard Error Logs:\n" + for f in stderr_files: + size = f.stat().st_size + result += f" {f.name}\n" + result += f" Size: {size:,} bytes\n" + result += f" Path: {f}\n" + result += "\n" + + return [TextContent(type="text", text=result)] + + except Exception as e: + return [TextContent( + type="text", + text=f"Error listing logs: {str(e)}" + )] + + elif name == "read_log": + log_path = arguments["log_path"] + tail = arguments.get("tail") + head = arguments.get("head") + grep = arguments.get("grep") + + try: + path = Path(log_path) + if not path.exists(): + return [TextContent( + type="text", + text=f"Error: Log file '{log_path}' does not exist" + )] + + # Read the file + with open(path, 'r', errors='replace') as f: + lines = f.readlines() + + total_lines = len(lines) + + # Apply filters + if grep: + lines = [line for line in lines if grep in line] + result = f"Log file: {path.name}\n" + result += f"Filtered by: '{grep}'\n" + result += f"Matching lines: {len(lines)} / {total_lines}\n" + result += "=" * 60 + "\n" + else: + result = f"Log file: {path.name}\n" + result += f"Total lines: {total_lines}\n" + result += "=" * 60 + "\n" + + if tail: + lines = lines[-tail:] + result += f"(showing last {tail} lines)\n\n" + elif head: + lines = lines[:head] + result += f"(showing first {head} lines)\n\n" + elif not grep: + # Default: show first 100 lines if no filter specified + if len(lines) > 100: + lines = lines[:100] + result += f"(showing first 100 of {total_lines} lines)\n\n" + + result += "".join(lines) + + # Warn if file was very large + if total_lines > 10000 and not (tail or head or grep): + result += f"\n\n[Note: File has {total_lines} lines. " + result += "Consider using --tail, --head, or --grep to filter]" + + return [TextContent(type="text", text=result)] + + except Exception as e: + return [TextContent( + type="text", + text=f"Error reading log file: {str(e)}" + )] + else: return [TextContent( type="text",
Signature · 1 line
2.55.0
PATCH 4/23 rteval: mcp-server: Add Claude Code plugin configuration
In reply to John KacurAdd plugin manifest and configuration files to enable the rteval MCP server as a Claude Code plugin. This allows conversational querying and analysis of rteval test results through Claude Code.
Changes:
- Add .claude-plugin/plugin.json with MCP server configuration
- Add manifest.json with plugin metadata
- Add test_tools.py to validate all MCP tools work correctly
- Update README.md with installation and usage instructions
The plugin provides 5 tools for rteval result analysis:
- list_results: Find rteval XML result files
- parse_result: Extract metrics from a single result
- compare_results: Compare two test runs with deltas
- list_logs: List stdout/stderr logs from a test run
- read_log: Read logs with head/tail/grep filtering
After restart, tools are available with the prefix:
mcp__plugin_rteval-mcp_rteval__
--- mcp-server/.claude-plugin/plugin.json | 20 +++++++++++++++ mcp-server/README.md | 37 +++++++++++++++++++++++++-- mcp-server/manifest.json | 20 +++++++++++++++ 3 files changed, 75 insertions(+), 2 deletions(-) create mode 100644 mcp-server/.claude-plugin/plugin.json create mode 100644 mcp-server/manifest.json
mcp-server/.claude-plugin/plugin.json new file +20 -0
@@ -0,0 +1,20 @@ +{ + "name": "rteval-mcp", + "description": "MCP server for analyzing rteval (real-time evaluation) test results", + "author": { + "name": "John Kacur", + "email": "[email protected]" + }, + "repository": "https://github.com/jkacur/rteval", + "license": "GPL-2.0", + "keywords": ["rteval", "real-time", "latency", "testing", "timerlat", "cyclictest"], + "mcpServers": { + "rteval": { + "command": "python3", + "args": [ + "/home/jkacur/src/rteval/mcp-server/server.py" + ], + "env": {} + } + } +}
mcp-server/README.md +35 -2
@@ -26,13 +26,46 @@ The initial focus is on multi-run analysis rather than single-result querying. ## Installation +On Fedora: ```bash -pip install -r requirements.txt +sudo dnf install python3-mcp python3-mcp+cli python3-lxml ``` +## Available Tools + +- **list_results**: List rteval result files in a directory +- **parse_result**: Parse an rteval XML result file and extract key metrics +- **compare_results**: Compare two rteval result files side-by-side +- **list_logs**: List available log files in an rteval result directory +- **read_log**: Read log content with optional filtering (head/tail/grep) + ## Usage -TBD - Server configuration and client usage instructions to be added. +### Testing the Server Directly + +```bash +python3 test_tools.py +``` + +### Using with Claude Code + +The server is configured as a Claude Code plugin. After installation: + +1. The server is registered in `~/.claude/plugins/installed_plugins.json` +2. Enabled in `~/.claude/settings.json` +3. Restart Claude Code to load the plugin +4. Tools will be available with the prefix: `mcp__plugin_rteval-mcp_rteval__` + +Example: +``` +list rteval results in ~/src/rteval +``` + +### Using with MCP Inspector + +```bash +mcp dev server.py +``` ## Development
mcp-server/manifest.json new file +20 -0
@@ -0,0 +1,20 @@ +{ + "manifest_version": "0.2", + "name": "rteval", + "version": "0.1.0", + "description": "rteval test results analysis - query and compare real-time evaluation test results", + "author": { + "name": "John Kacur", + "email": "[email protected]" + }, + "server": { + "type": "python", + "entry_point": "server.py", + "mcp_config": { + "command": "python3", + "args": [ + "/home/jkacur/src/rteval/mcp-server/server.py" + ] + } + } +}
Signature · 1 line
2.55.0
PATCH 5/23 rteval: Add .mcp.json for project-level MCP server configuration
In reply to John KacurAdd .mcp.json to configure the rteval MCP server for the project. This enables the server to be automatically discovered and used by Claude Code when working in the rteval repository.
The MCP server provides conversational access to rteval test results:
- Query and parse XML result files
- Compare multiple test runs
- Access log files with filtering
Server is invoked as "rteval" and tools are prefixed with mcp__rteval__
--- .mcp.json | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 .mcp.json
.mcp.json new file +11 -0
@@ -0,0 +1,11 @@ +{ + "mcpServers": { + "rteval": { + "command": "python3", + "args": [ + "mcp-server/server.py" + ], + "env": {} + } + } +}
Signature · 1 line
2.55.0
PATCH 6/23 rteval: mcp-server: Add rtla command line parser
In reply to John KacurAdd parse_rtla_command() function to parse rtla timerlat and cyclictest command line parameters. Update parse_result tool to display parsed parameters in a human-readable format, showing period, scheduling, CPUs, thresholds, and other settings.
Also update .mcp.json to use absolute path for better reliability.
--- .mcp.json | 2 +- mcp-server/server.py | 140 ++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 139 insertions(+), 3 deletions(-)
.mcp.json +1 -1
@@ -3,7 +3,7 @@ "rteval": { "command": "python3", "args": [ - "mcp-server/server.py" + "/home/jkacur/src/rteval/mcp-server/server.py" ], "env": {} }
mcp-server/server.py +138 -2
158 lines not on this page · raw message
Signature · 1 line
2.55.0
Re: [PATCH 06/23] rteval: mcp-server: Add rtla command line parser
so 15. 8. 2026 v 0:11 odesílatel John Kacur <[email protected]> napsal:
Quoted 20 lines
Add
parse_rtla_command()function to parse rtla timerlat and cyclictest command line parameters. Updateparse_resulttool to display parsed parameters in a human-readable format, showing period, scheduling, CPUs, thresholds, and other settings.Also update .mcp.json to use absolute path for better reliability.
Assisted-by Claude:claude-sonnet-4-5Signed-off-by John Kacur <[email protected]>--- .mcp.json | 2 +- mcp-server/server.py | 140 ++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 139 insertions(+), 3 deletions(-).mcp.json +0
-0
.mcp.json line 7
Quoted 6 lines
.mcp.json +1
-1@@ -3,7 +3,7 @@ "rteval": { "command": "python3", "args": [ - "mcp-server/server.py" + "/home/jkacur/src/rteval/mcp-server/server.py" ],
I certainly don't have "/home/jkacur" on my system. This looks like a leftover artifact from a Claude debugging session.
Quoted 1 line
[truncated]
Tomas
Re: [PATCH 06/23] mcp-server: Add plugin.json configuration
Hi Tomas,
Quoted 2 lines · Tomas Glozar
I certainly don't have "
/home/jkacur" on my system. This looks like a leftover artifact from a Claude debugging session.
You're right that the absolute path won't work on other systems. The path shown here is for running the MCP server directly from git during development, but it's specific to my checkout location.
This is fixed later in the series:
Patch 22/23 (002445fe017d "mcp-server: Restructure as Python package with dynamic paths") restructures the code as a proper Python package and introduces a wrapper script that dynamically calculates paths based on its own location.
Patch 23/23 (c16aa3b26492 "mcp-server: Use relative path in .mcp.json and add example for installation") then updates .mcp.json to use a relative path:
./rteval_mcp/rteval-mcp-server
This allows the MCP server to run from any git checkout location without requiring modification, and the wrapper script handles both development (git checkout) and system installation scenarios.
Thanks for reviewing!
John
Re: [PATCH 06/23] mcp-server: Add plugin.json configuration
Hi John,
po 17. 8. 2026 v 22:59 odesílatel John Kacur <[email protected]> napsal:
Quoted 11 lines
Hi Tomas,
Quoted 2 lines · Tomas Glozar
I certainly don't have "
/home/jkacur" on my system. This looks like a leftover artifact from a Claude debugging session.You're right that the absolute path won't work on other systems. The path shown here is for running the MCP server directly from git during development, but it's specific to my checkout location.
Quoted 7 lines
This is fixed later in the series:
Patch 22/23 (
002445fe017d"mcp-server: Restructure as Python package with dynamic paths") restructures the code as a proper Python package and introduces a wrapper script that dynamically calculates paths based on its own location.
Ah, I see. Sorry for the noise.
Quoted 6 lines
Patch 23/23 (
c16aa3b26492"mcp-server: Use relative path in .mcp.json and add example for installation") then updates .mcp.json to use a relative path:./rteval_mcp/rteval-mcp-server
Quoted 8 lines
This allows the MCP server to run from any git checkout location without requiring modification, and the wrapper script handles both development (git checkout) and system installation scenarios.
Thanks for reviewing!
John
You might want to consider cleaning up patchsets to exclude development artifacts like this, so that reviewer's work is easier. In this case for example, you add an MCP server, then restructure it in the same patchset. The reviewer would ideally only review the final result.
Tomas
PATCH 7/23 rteval: mcp-server: Add batch analysis tool
In reply to John KacurAdd batch_analysis tool to analyze multiple rteval result files at once. The tool scans directories for result files, parses each one, and aggregates statistics across all runs.
Features:
- Directory scanning with pattern matching (*.xml by default)
- Optional recursive subdirectory search
- Individual file summaries showing date, kernel, duration, and latencies
- Aggregate statistics: date range, max/mean latencies (min/max/average)
- Error handling for unparseable files
This enables quick analysis of multiple test runs to identify trends, compare results over time, and spot outliers.
--- mcp-server/server.py | 174 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 174 insertions(+)
mcp-server/server.py +174 -0
@@ -273,6 +273,30 @@ async def list_tools() -> list[Tool]: "required": ["log_path"], }, ), + Tool( + name="batch_analysis", + description="Analyze multiple rteval result files and aggregate statistics", + inputSchema={ + "type": "object", + "properties": { + "directory": { + "type": "string", + "description": "Directory to search for rteval results (default: current directory)", + }, + "pattern": { + "type": "string", + "description": "File pattern to match (default: *.xml)", + "default": "*.xml", + }, + "recursive": { + "type": "boolean", + "description": "Search subdirectories recursively (default: false)", + "default": False, + }, + }, + "required": [], + }, + ), ]
@@ -715,6 +739,156 @@ async def call_tool(name: str, arguments: Any) -> list[TextContent]: text=f"Error reading log file: {str(e)}" )] + elif name == "batch_analysis": + directory = arguments.get("directory", ".") + pattern = arguments.get("pattern", "*.xml") + recursive = arguments.get("recursive", False) + + try: + path = Path(directory) + if not path.exists(): + return [TextContent( + type="text", + text=f"Error: Directory '{directory}' does not exist" + )] + + # Find matching files + if recursive: + files = list(path.rglob(pattern)) + else: + files = list(path.glob(pattern)) + + if not files: + return [TextContent( + type="text", + text=f"No files matching '{pattern}' found in '{directory}'" + )] + + # Parse each file and collect data + results = [] + parse_errors = [] + + for file_path in files: + try: + data = extract_rteval_data(str(file_path)) + results.append(data) + except Exception as e: + parse_errors.append(f"{file_path.name}: {str(e)}") + + if not results: + error_msg = "Failed to parse any result files" + if parse_errors: + error_msg += ":\n" + "\n".join(parse_errors) + return [TextContent(type="text", text=error_msg)] + + # Build output + result = f"Batch Analysis of {len(results)} rteval Result(s)\n" + result += "=" * 60 + "\n\n" + + if parse_errors: + result += f"Warning: Failed to parse {len(parse_errors)} file(s)\n\n" + + # Individual file summaries + result += "Individual Results:\n" + result += "-" * 60 + "\n" + + max_latencies = [] + mean_latencies = [] + dates = [] + + for data in results: + file_name = Path(data["file"]).name + result += f"\n{file_name}:\n" + + # Date + if "date" in data["run_info"] and "time" in data["run_info"]: + date_str = f"{data['run_info']['date']} {data['run_info']['time']}" + result += f" Date: {date_str}\n" + dates.append(date_str) + + # Duration + ri = data["run_info"] + if any(k in ri for k in ["days", "hours", "minutes", "seconds"]): + result += f" Duration: {ri.get('days', 0)}d {ri.get('hours', 0)}h " + result += f"{ri.get('minutes', 0)}m {ri.get('seconds', 0)}s\n" + + # System + if "kernel" in data["system_info"]: + result += f" Kernel: {data['system_info']['kernel']}" + if "is_RT" in data["system_info"]: + result += f" (RT: {'Yes' if data['system_info']['is_RT'] else 'No'})" + result += "\n" + + # Measurements - timerlat + if "timerlat" in data["measurements"]: + meas = data["measurements"]["timerlat"] + if "maximum" in meas: + max_val = meas["maximum"]["value"] + max_unit = meas["maximum"]["unit"] + result += f" Max Latency: {max_val} {max_unit}\n" + try: + max_latencies.append(float(max_val)) + except ValueError: + pass + if "mean" in meas: + mean_val = meas["mean"]["value"] + mean_unit = meas["mean"]["unit"] + result += f" Mean Latency: {mean_val} {mean_unit}\n" + try: + mean_latencies.append(float(mean_val)) + except ValueError: + pass + + # Measurements - cyclictest + elif "cyclictest" in data["measurements"]: + meas = data["measurements"]["cyclictest"] + if "maximum" in meas: + max_val = meas["maximum"]["value"] + max_unit = meas["maximum"]["unit"] + result += f" Max Latency: {max_val} {max_unit}\n" + try: + max_latencies.append(float(max_val)) + except ValueError: + pass + if "mean" in meas: + mean_val = meas["mean"]["value"] + mean_unit = meas["mean"]["unit"] + result += f" Mean Latency: {mean_val} {mean_unit}\n" + try: + mean_latencies.append(float(mean_val)) + except ValueError: + pass + + # Aggregate statistics + result += "\n" + "=" * 60 + "\n" + result += "Aggregate Statistics:\n" + result += "-" * 60 + "\n\n" + + result += f"Total files analyzed: {len(results)}\n" + + if dates: + result += f"Date range: {min(dates)} to {max(dates)}\n" + + if max_latencies: + result += f"\nMaximum Latencies:\n" + result += f" Lowest max: {min(max_latencies):.2f} µs\n" + result += f" Highest max: {max(max_latencies):.2f} µs\n" + result += f" Average max: {sum(max_latencies) / len(max_latencies):.2f} µs\n" + + if mean_latencies: + result += f"\nMean Latencies:\n" + result += f" Lowest mean: {min(mean_latencies):.2f} µs\n" + result += f" Highest mean: {max(mean_latencies):.2f} µs\n" + result += f" Average mean: {sum(mean_latencies) / len(mean_latencies):.2f} µs\n" + + return [TextContent(type="text", text=result)] + + except Exception as e: + return [TextContent( + type="text", + text=f"Error in batch analysis: {str(e)}" + )] + else: return [TextContent( type="text",
Signature · 1 line
2.55.0
PATCH 8/23 rteval: mcp-server: Add query and filter capabilities
In reply to John KacurAdd three new tools to the MCP server for advanced result analysis:
filter_results - Filter rteval results by multiple criteria including kernel version pattern, RT kernel status, date range, minimum test duration, and maximum latency threshold. Enables targeted analysis of specific test configurations.
find_best_worst - Identify optimal and poorest performing runs based on configurable latency metrics (maximum, mean, or median). Returns ranked results with kernel version, date, and latency information.
compare_to_baseline - Regression detection by comparing multiple results against a baseline file. Detects and reports regressions above a configurable threshold percentage (default 10%), showing detailed percentage changes for all metrics.
Updated README with tool descriptions, usage examples, and detailed feature documentation to help users understand the new capabilities.
--- mcp-server/README.md | 81 +++++++- mcp-server/server.py | 450 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 530 insertions(+), 1 deletion(-)
mcp-server/README.md +80 -1
@@ -33,9 +33,18 @@ sudo dnf install python3-mcp python3-mcp+cli python3-lxml ## Available Tools +### Basic Analysis - **list_results**: List rteval result files in a directory - **parse_result**: Parse an rteval XML result file and extract key metrics - **compare_results**: Compare two rteval result files side-by-side +- **batch_analysis**: Analyze multiple rteval result files and aggregate statistics + +### Query & Filter +- **filter_results**: Filter rteval results by kernel version, date range, test duration, and latency thresholds +- **find_best_worst**: Find best and worst rteval runs based on latency metrics (maximum, mean, or median) +- **compare_to_baseline**: Compare multiple results against a baseline file and detect regressions + +### Log Access - **list_logs**: List available log files in an rteval result directory - **read_log**: Read log content with optional filtering (head/tail/grep)
@@ -56,9 +65,40 @@ The server is configured as a Claude Code plugin. After installation: 3. Restart Claude Code to load the plugin 4. Tools will be available with the prefix: `mcp__plugin_rteval-mcp_rteval__` -Example: +Example queries: ``` list rteval results in ~/src/rteval +find the best and worst rteval runs +show me rteval results with kernel 7.0 +compare rteval results to a baseline +``` + +### Query & Filter Examples + +**Find best and worst runs:** +``` +Find the 3 best and worst rteval runs by maximum latency +Which rteval runs had the lowest mean latency? +``` + +**Filter by criteria:** +``` +Show me rteval results with kernel version 7.0 +Find rteval results from June 2026 +Show results with max latency under 3500 microseconds +Filter rteval results by RT kernel +``` + +**Baseline comparison:** +``` +Compare all rteval results to rteval-20260605-1/summary.xml +Check for regressions against the baseline with 5% threshold +``` + +**Batch analysis:** +``` +Analyze all rteval results and show aggregate statistics +What's the average max latency across all runs? ``` ### Using with MCP Inspector
@@ -67,6 +107,45 @@ list rteval results in ~/src/rteval mcp dev server.py ``` +## Features in Detail + +### filter_results +Filter rteval results by multiple criteria: +- **kernel_pattern**: Match kernel version (e.g., "7.0", "rt", "6.17") +- **is_rt**: Filter by RT kernel (true/false) +- **date_from/date_to**: Date range filter (YYYY-MM-DD format) +- **min_duration_minutes**: Minimum test duration +- **max_latency_threshold**: Maximum acceptable latency in µs + +Example: Find all RT kernel runs from June 2026 with max latency under 4000 µs + +### find_best_worst +Identify optimal and poorest performing runs: +- **metric**: Choose maximum, mean, or median latency +- **count**: Number of best/worst results to return (default: 5) + +Shows ranked results with kernel version, date, and latency metrics. + +### compare_to_baseline +Regression detection against a baseline result: +- **baseline_file**: Reference rteval XML file +- **directory**: Directory containing results to compare +- **threshold_percent**: Alert threshold for regressions (default: 10%) + +Reports: +- Regressions (results exceeding threshold) +- Passes (results within threshold) +- Detailed percentage changes for all metrics + +### batch_analysis +Aggregate statistics across multiple runs: +- Total files analyzed +- Date range coverage +- Min/max/average latencies across all runs +- Individual result summaries + +Useful for understanding performance trends over time. + ## Development This is a work in progress. The MCP server is being developed in a branch of the
mcp-server/server.py +450 -0
462 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 9/23 rteval MCP: Add histogram extraction and percentile analysis
In reply to John KacurImplement feature #3 from future-features.md: latency histogram extraction and percentile calculation.
New tools:
- extract_histogram: Extract raw histogram data from rteval XML
- System-wide and per-CPU histogram summaries
- Top latency values by sample count
- Total samples and latency range
- get_percentiles: Calculate latency percentiles (P50, P90, P95, P99, P99.9, P99.99)
- System-wide percentile analysis
- Optional per-CPU percentile breakdown
- Reveals latency distribution beyond max/mean/median
Why this matters:
- Max latency alone is misleading (one spike vs systematic issues)
- P99.9 tells you 99.9% of samples were below this value
- Percentiles distinguish rare outliers from regular tail latency
- Per-CPU percentiles identify hardware-specific problems
Implementation:
- extract_histogram_data(): Parse histogram buckets from XML
- calculate_percentiles(): Compute percentiles from bucket counts
- Fixed XPath to find system-wide histogram under <system> tag
- Handles both system-wide and per-CPU histogram data
Testing:
- test_histogram.py validates extraction and calculation
- Tested with rteval-20260714-1 (28.6M samples, 16 CPUs)
- System P99.9 = 16µs, max = 1550µs (excellent with rare outliers)
Documentation:
- Updated README with usage examples
- Added detailed feature descriptions
- Explained why percentiles matter for RT analysis
--- mcp-server/README.md | 40 +++++ mcp-server/server.py | 275 +++++++++++++++++++++++++++++++++++ mcp-server/test_histogram.py | 57 ++++++++ 3 files changed, 372 insertions(+) create mode 100644 mcp-server/test_histogram.py
mcp-server/README.md +40 -0
@@ -48,6 +48,10 @@ sudo dnf install python3-mcp python3-mcp+cli python3-lxml - **list_logs**: List available log files in an rteval result directory - **read_log**: Read log content with optional filtering (head/tail/grep) +### Histogram Analysis +- **extract_histogram**: Extract latency histogram data from rteval results +- **get_percentiles**: Calculate latency percentiles (P50, P95, P99, P99.9, etc.) + ## Usage ### Testing the Server Directly
@@ -101,6 +105,14 @@ Analyze all rteval results and show aggregate statistics What's the average max latency across all runs? ``` +**Histogram and percentiles:** +``` +Extract histogram data from rteval-20260714-1/summary.xml +Calculate P99 and P99.9 percentiles for rteval-20260714-1/summary.xml +Show me percentiles for each CPU in rteval-20260714-1/summary.xml +What percentage of samples had latency under 10 microseconds? +``` + ### Using with MCP Inspector ```bash
@@ -146,6 +158,34 @@ Aggregate statistics across multiple runs: Useful for understanding performance trends over time. +### extract_histogram +Extract raw histogram data from rteval results: +- **file_path**: Path to rteval XML result file +- **include_per_cpu**: Include per-CPU histogram data (default: true) + +Shows: +- System-wide histogram summary with total samples and latency range +- Top latency values by sample count with percentages +- Per-CPU histogram summaries with individual max latencies + +### get_percentiles +Calculate latency percentiles for detailed distribution analysis: +- **file_path**: Path to rteval XML result file +- **percentiles**: List of percentiles to calculate (default: [50, 90, 95, 99, 99.9, 99.99]) +- **per_cpu**: Calculate percentiles per CPU (default: false) + +Reports: +- System-wide percentiles showing latency distribution +- Per-CPU percentiles when requested for fine-grained analysis + +**Why percentiles matter:** +- **Max latency** alone can be misleading (one outlier vs. systematic issues) +- **P99.9** tells you that 99.9% of samples were below this value +- **Percentiles** reveal whether high latencies are rare spikes or regular occurrences +- **Per-CPU percentiles** help identify hardware-specific issues or IRQ affinity problems + +Example: If P99.9 = 16µs but max = 1550µs, this tells you the system is excellent with only rare outliers, not a systematic problem. + ## Development This is a work in progress. The MCP server is being developed in a branch of the
mcp-server/server.py +275 -0
293 lines not on this page · raw message
mcp-server/test_histogram.py new file +57 -0
57 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 10/23 rteval MCP: Reorganize tests into tests/ subdirectory
In reply to John KacurMove test scripts from mcp-server/ to mcp-server/tests/ for better organization following Python conventions.
Changes:
- Create tests/ directory
- Move test_histogram.py to tests/
- Move test_new_tools.py to tests/
- Update import paths to handle parent directory
- Update file paths to work from any execution location
Tests can now be run from:
- rteval/ : python3 mcp-server/tests/test_histogram.py
- rteval/mcp-server/ : python3 tests/test_histogram.py
- rteval/mcp-server/tests/ : python3 test_histogram.py
Implementation:
- Use sys.path.insert(0, str(Path(__file__).parent.parent)) to find server.py
- Use Path(__file__).parent.parent.parent for data file resolution
- Paths are resolved relative to script location, not cwd
Tested from all three directories successfully.
---
mcp-server/{ => tests}/test_histogram.py | 13 ++-
mcp-server/tests/test_new_tools.py | 135 +++++++++++++++++++++++
2 files changed, 145 insertions(+), 3 deletions(-)
rename mcp-server/{ => tests}/test_histogram.py (82%)
create mode 100644 mcp-server/tests/test_new_tools.pymcp-server/tests/test_histogram.py renamed from mcp-server/test_histogram.py +10 -3
@@ -3,16 +3,23 @@ Test script for histogram extraction and percentile calculation. """ +import sys +from pathlib import Path + +# Add parent directory to path to import server module +sys.path.insert(0, str(Path(__file__).parent.parent)) from server import extract_histogram_data, calculate_percentiles -# Test with the sample file -test_file = "../rteval-20260714-1/summary.xml" +# Find test file relative to script location +# From tests/ -> mcp-server/ -> rteval/ +script_dir = Path(__file__).parent +test_file = script_dir.parent.parent / "rteval-20260714-1" / "summary.xml" print("Testing histogram extraction...") print("=" * 60) # Extract histogram data -histogram_data = extract_histogram_data(test_file) +histogram_data = extract_histogram_data(str(test_file)) # Check system histogram if histogram_data["system_histogram"]:
mcp-server/tests/test_new_tools.py new file +135 -0
135 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 11/23 rteval MCP: Fix test_new_tools.py to find rteval result files
In reply to John KacurUpdate glob pattern from "*.xml" to "rteval-*/summary.xml" to correctly locate rteval result files in their subdirectories.
The original test looked for XML files directly in the top-level directory, but rteval stores results in subdirectories like rteval-20260714-1/summary.xml.
Changes:
- test_find_best_worst: Use rteval-*/summary.xml pattern
- test_filter_results: Use rteval-*/summary.xml pattern
Testing:
- Successfully finds 13 rteval result files
- Works from both rteval/ and rteval/mcp-server/ directories
- Shows best/worst results correctly (1355µs to 3499µs range)
- Filters by kernel version correctly (12 of 13 match "7.0")
--- mcp-server/tests/test_new_tools.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mcp-server/tests/test_new_tools.py +2 -2
@@ -21,7 +21,7 @@ async def test_find_best_worst(): metric = "maximum" count = 3 - files = list(directory.glob("*.xml")) + files = list(directory.glob("rteval-*/summary.xml")) print(f"Found {len(files)} XML files")
@@ -84,7 +84,7 @@ async def test_filter_results(): directory = Path(__file__).parent.parent.parent kernel_pattern = "7.0" - files = list(directory.glob("*.xml")) + files = list(directory.glob("rteval-*/summary.xml")) print(f"Filtering for kernel pattern: {kernel_pattern}") print(f"Searching {len(files)} XML files\n")
Signature · 1 line
2.55.0
PATCH 12/23 rteval MCP: Fix glob patterns to find rteval result files
In reply to John KacurThree MCP tools were using "*.xml" pattern which doesn't find rteval result files in their subdirectories. Changed to "rteval-*/summary.xml" to correctly locate rteval results.
Fixed tools:
- filter_results: Now finds rteval result files for kernel/date filtering
- find_best_worst: Now finds rteval result files for best/worst analysis
- compare_to_baseline: Now finds rteval result files for regression detection
The batch_analysis tool already had a configurable pattern parameter and was working correctly.
This bug was discovered by the new integration test which validates that all MCP tools work end-to-end through the MCP interface.
--- mcp-server/server.py | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
mcp-server/server.py +8 -8
@@ -1149,12 +1149,12 @@ async def call_tool(name: str, arguments: Any) -> list[TextContent]: text=f"Error: Directory '{directory}' does not exist" )] - # Find all XML files - files = list(path.glob("*.xml")) + # Find rteval result files + files = list(path.glob("rteval-*/summary.xml")) if not files: return [TextContent( type="text", - text=f"No XML files found in '{directory}'" + text=f"No rteval result files found in '{directory}'" )] # Parse and filter
@@ -1262,12 +1262,12 @@ async def call_tool(name: str, arguments: Any) -> list[TextContent]: text=f"Error: Directory '{directory}' does not exist" )] - # Find all XML files - files = list(path.glob("*.xml")) + # Find rteval result files + files = list(path.glob("rteval-*/summary.xml")) if not files: return [TextContent( type="text", - text=f"No XML files found in '{directory}'" + text=f"No rteval result files found in '{directory}'" )] # Parse and collect metrics
@@ -1380,11 +1380,11 @@ async def call_tool(name: str, arguments: Any) -> list[TextContent]: text=f"Error: Directory '{directory}' does not exist" )] - files = list(path.glob("*.xml")) + files = list(path.glob("rteval-*/summary.xml")) if not files: return [TextContent( type="text", - text=f"No XML files found in '{directory}'" + text=f"No rteval result files found in '{directory}'" )] # Build comparison report
Signature · 1 line
2.55.0
PATCH 13/23 rteval MCP: Add integration tests and remove hardcoded examples
In reply to John KacurReplace hardcoded example scripts with a proper integration test that validates MCP server tools through the actual MCP interface.
Removed files (had hardcoded paths to deleted test data):
- test_server.py: Basic sanity check, too minimal
- test_comparison.py: Redundant with compare_results tool
- test_logs.py: Redundant with list_logs/read_log tools
- test_tools.py: Replaced by test_mcp_integration.py
Added file:
- tests/test_mcp_integration.py: End-to-end integration test
- Tests 8 MCP tools through the MCP interface
- Dynamically finds available rteval result directories
- No hardcoded paths - works with any rteval data
- Works from rteval/, rteval/mcp-server/, or rteval/mcp-server/tests/
- Discovered bugs in filter_results, find_best_worst, compare_to_baseline
Test coverage:
- list_tools, list_results, parse_result, list_logs
- get_percentiles, extract_histogram (new histogram features)
- filter_results, find_best_worst (query/filter tools)
All tests pass after server.py glob pattern fixes.
--- mcp-server/tests/test_mcp_integration.py | 195 +++++++++++++++++++++++ 1 file changed, 195 insertions(+) create mode 100644 mcp-server/tests/test_mcp_integration.py
mcp-server/tests/test_mcp_integration.py new file +195 -0
@@ -0,0 +1,195 @@ +#!/usr/bin/env python3 +"""Integration tests for rteval MCP server. + +Tests the MCP server tools through the actual MCP interface, +validating end-to-end functionality. +""" + +import sys +import asyncio +from pathlib import Path + +# Add parent directory to path to import server module +sys.path.insert(0, str(Path(__file__).parent.parent)) +from server import call_tool, list_tools + + +def find_test_data(): + """Find available rteval result directories for testing.""" + # From tests/ -> mcp-server/ -> rteval/ + rteval_dir = Path(__file__).parent.parent.parent + + # Find rteval result directories (not rteval-build) + result_dirs = sorted([ + d for d in rteval_dir.glob("rteval-*/") + if d.is_dir() and d.name != "rteval-build" and (d / "summary.xml").exists() + ]) + + if not result_dirs: + raise RuntimeError("No rteval result directories found for testing") + + return result_dirs + + +async def main(): + print("MCP Server Integration Tests") + print("=" * 60) + + # Find test data + try: + result_dirs = find_test_data() + test_dir = result_dirs[0] + test_file = test_dir / "summary.xml" + rteval_parent = test_dir.parent + + print(f"\nUsing test data: {test_dir.name}") + print(f"Total result directories available: {len(result_dirs)}") + print() + except RuntimeError as e: + print(f"✗ {e}") + sys.exit(1) + + # Test 1: List available tools + print("1. Testing list_tools()...") + try: + tools = await list_tools() + print(f"✓ Found {len(tools)} tools:") + for tool in tools: + print(f" - {tool.name}: {tool.description[:60]}...") + except Exception as e: + print(f"✗ Failed: {e}") + sys.exit(1) + + # Test 2: List results + print("\n2. Testing list_results...") + try: + result = await call_tool("list_results", { + "directory": str(rteval_parent), + "pattern": "rteval-*/summary.xml" + }) + output = result[0].text + if "Found" in output and "file" in output: + lines = output.split('\n')[:5] + print("✓ " + '\n '.join(lines)) + else: + print(f"✗ Unexpected output: {output[:200]}") + sys.exit(1) + except Exception as e: + print(f"✗ Failed: {e}") + sys.exit(1) + + # Test 3: Parse a result + print("\n3. Testing parse_result...") + try: + result = await call_tool("parse_result", { + "file_path": str(test_file) + }) + output = result[0].text + if "rteval version" in output and "System Information" in output: + lines = output.split('\n')[:10] + print("✓ " + '\n '.join(lines)) + else: + print(f"✗ Unexpected output: {output[:200]}") + sys.exit(1) + except Exception as e: + print(f"✗ Failed: {e}") + sys.exit(1) + + # Test 4: List logs + print("\n4. Testing list_logs...") + try: + result = await call_tool("list_logs", { + "result_dir": str(test_dir) + }) + output = result[0].text + if "Log files" in output or "logs" in output.lower(): + lines = output.split('\n')[:8] + print("✓ " + '\n '.join(lines)) + else: + print(f"✗ Unexpected output: {output[:200]}") + sys.exit(1) + except Exception as e: + print(f"✗ Failed: {e}") + sys.exit(1) + + # Test 5: Get percentiles (new histogram feature) + print("\n5. Testing get_percentiles...") + try: + result = await call_tool("get_percentiles", { + "file_path": str(test_file), + "percentiles": [50, 95, 99, 99.9] + }) + output = result[0].text + if "Percentiles" in output and "P50" in output: + lines = output.split('\n')[:12] + print("✓ " + '\n '.join(lines)) + else: + print(f"✗ Unexpected output: {output[:200]}") + sys.exit(1) + except Exception as e: + print(f"✗ Failed: {e}") + sys.exit(1) + + # Test 6: Extract histogram (new feature) + print("\n6. Testing extract_histogram...") + try: + result = await call_tool("extract_histogram", { + "file_path": str(test_file), + "include_per_cpu": False # Keep output brief + }) + output = result[0].text + if "Histogram Data" in output and "Total Samples" in output: + lines = output.split('\n')[:12] + print("✓ " + '\n '.join(lines)) + else: + print(f"✗ Unexpected output: {output[:200]}") + sys.exit(1) + except Exception as e: + print(f"✗ Failed: {e}") + sys.exit(1) + + # Test 7: Filter results (query tools) + print("\n7. Testing filter_results...") + try: + result = await call_tool("filter_results", { + "directory": str(rteval_parent), + "is_rt": False + }) + output = result[0].text + if "Filtered Results" in output or "files matched" in output: + lines = output.split('\n')[:8] + print("✓ " + '\n '.join(lines)) + else: + print(f"✗ Unexpected output: {output[:200]}") + sys.exit(1) + except Exception as e: + print(f"✗ Failed: {e}") + sys.exit(1) + + # Test 8: Find best/worst + print("\n8. Testing find_best_worst...") + try: + result = await call_tool("find_best_worst", { + "directory": str(rteval_parent), + "metric": "maximum", + "count": 3 + }) + output = result[0].text + if ("BEST" in output and "WORST" in output) or "No results" in output: + lines = output.split('\n')[:10] + print("✓ " + '\n '.join(lines)) + else: + print(f"✗ Unexpected output: {output[:200]}") + sys.exit(1) + except Exception as e: + print(f"✗ Failed: {e}") + sys.exit(1) + + print("\n" + "=" * 60) + print("✓ All MCP server integration tests passed!") + print(f"\nTested with: {test_dir.name}") + print(f"Total tools tested: 8") + + +if __name__ == "__main__": + asyncio.run(main())
Signature · 1 line
2.55.0
PATCH 14/23 rteval MCP: Add per-CPU latency statistics breakdown
In reply to John KacurAdd get_per_cpu_stats tool to provide detailed per-CPU latency statistics from rteval summary.xml files, enabling granular performance analysis across individual CPU cores.
Features:
- Extract latency statistics for individual CPU cores
- Filter by specific CPU IDs or ranges (e.g., "4-8")
- Show all CPUs when no filter specified
- Return statistics for both measurement module and system overall
- Support for cyclictest and timerlat measurement modules
Statistics per CPU:
- Sample count
- Min, max, mean, median, mode latencies
- Standard deviation and mean absolute deviation
- Full latency range
Return format:
- measurement_module: Module name (cyclictest/timerlat)
- system_stats: Overall system statistics
- per_cpu_stats: Array of statistics per CPU
- cpu_id: CPU identifier
- priority: Thread priority
- samples, min, max, mean, median, mode, range
- mean_absolute_deviation, standard_deviation
Use cases:
- Identify outlier CPUs with poor latency
- Compare latency across CPU cores
- Analyze specific CPU subsets
- Validate CPU isolation effectiveness
Updated README with detailed tool documentation and usage examples.
Added comprehensive test coverage in test_per_cpu_stats.py.
--- mcp-server/README.md | 37 ++++- mcp-server/server.py | 169 +++++++++++++++++++++++ mcp-server/tests/test_mcp_integration.py | 20 ++- mcp-server/tests/test_per_cpu_stats.py | 124 +++++++++++++++++ 4 files changed, 348 insertions(+), 2 deletions(-) create mode 100644 mcp-server/tests/test_per_cpu_stats.py
mcp-server/README.md +36 -1
@@ -48,9 +48,10 @@ sudo dnf install python3-mcp python3-mcp+cli python3-lxml - **list_logs**: List available log files in an rteval result directory - **read_log**: Read log content with optional filtering (head/tail/grep) -### Histogram Analysis +### Histogram & Per-CPU Analysis - **extract_histogram**: Extract latency histogram data from rteval results - **get_percentiles**: Calculate latency percentiles (P50, P95, P99, P99.9, etc.) +- **get_per_cpu_stats**: Get per-CPU latency statistics and identify problematic cores ## Usage
@@ -113,6 +114,15 @@ Show me percentiles for each CPU in rteval-20260714-1/summary.xml What percentage of samples had latency under 10 microseconds? ``` +**Per-CPU analysis:** +``` +Show me per-CPU statistics for rteval-20260714-1/summary.xml +Which CPUs had the worst maximum latency? +Show the top 5 CPUs sorted by mean latency +Highlight CPUs with max latency above 1000 microseconds +Which CPUs have the most variable latency? +``` + ### Using with MCP Inspector ```bash
@@ -186,6 +196,31 @@ Reports: Example: If P99.9 = 16µs but max = 1550µs, this tells you the system is excellent with only rare outliers, not a systematic problem. +### get_per_cpu_stats +Get detailed per-CPU latency statistics to identify problematic cores: +- **file_path**: Path to rteval XML result file +- **sort_by**: Metric to sort by - 'maximum', 'mean', 'median', or 'standard_deviation' (default: maximum) +- **show_top_n**: Show only top N worst CPUs (0 = show all, default: 0) +- **highlight_threshold**: Highlight CPUs with max latency above this threshold in µs (optional) + +Shows: +- Tabular view of all CPU statistics (samples, min, max, mean, median, standard deviation) +- CPUs sorted by worst performance first for the selected metric +- Warning indicators for CPUs exceeding the threshold +- Summary statistics across all CPUs + +**Why per-CPU analysis matters:** +- **Identify problematic cores**: Some CPUs may have significantly worse latency than others +- **Hardware issues**: Bad cores, thermal throttling, or IRQ affinity problems +- **NUMA effects**: CPUs on different sockets may show different behavior +- **CPU pinning**: Determine which CPUs are best for RT workloads + +Example use cases: +- "Which CPU had the worst latency spike?" → Sort by maximum +- "Which CPUs are most consistent?" → Sort by standard_deviation (low is better) +- "Are any CPUs consistently slow?" → Sort by mean latency +- "Show me only the 5 worst CPUs" → Use show_top_n=5 + ## Development This is a work in progress. The MCP server is being developed in a branch of the
mcp-server/server.py +169 -0
187 lines not on this page · raw message
mcp-server/tests/test_mcp_integration.py +19 -1
@@ -185,10 +185,28 @@ async def main(): print(f"✗ Failed: {e}") sys.exit(1) + # Test 9: Get per-CPU stats + print("\n9. Testing get_per_cpu_stats...") + try: + result = await call_tool("get_per_cpu_stats", { + "file_path": str(test_file), + "show_top_n": 5 + }) + output = result[0].text + if "Per-CPU Latency Statistics" in output and "CPU" in output: + lines = output.split('\n')[:12] + print("✓ " + '\n '.join(lines)) + else: + print(f"✗ Unexpected output: {output[:200]}") + sys.exit(1) + except Exception as e: + print(f"✗ Failed: {e}") + sys.exit(1) + print("\n" + "=" * 60) print("✓ All MCP server integration tests passed!") print(f"\nTested with: {test_dir.name}") - print(f"Total tools tested: 8") + print(f"Total tools tested: 9") if __name__ == "__main__":
mcp-server/tests/test_per_cpu_stats.py new file +124 -0
124 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 15/23 mcp-server: Add overflow bucket support and cyclictest compatibility
In reply to John KacurAdd support for the new overflow bucket feature in rteval XML output. Overflow buckets track latency samples that exceeded the configured histogram range (--buckets parameter).
Changes to extract_histogram_data():
- Handle overflow buckets to prevent ValueError when parsing index="overflow" attributes
- Add cyclictest support (was timerlat-only)
- Track overflow_count and overflow_bucket in extracted data
- Display overflow warnings when samples exceeded histogram range
- Support both system-wide and per-CPU overflow tracking
This fixes crashes when parsing rteval results with overflow samples and provides visibility into histogram sizing issues.
--- mcp-server/server.py | 55 +++++++++++++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 13 deletions(-)
mcp-server/server.py +42 -13
@@ -193,51 +193,74 @@ def extract_histogram_data(file_path: str) -> dict[str, Any]: "per_cpu_histograms": [] } - # Extract system-wide histogram - timerlat = root.find(".//timerlat") - if timerlat is not None: + # Extract system-wide histogram - try timerlat first, then cyclictest + measurement = root.find(".//timerlat") + if measurement is None: + measurement = root.find(".//cyclictest") + + if measurement is not None: # System-wide histogram is under <system> tag - system_elem = timerlat.find("system") + system_elem = measurement.find("system") system_histogram = None if system_elem is not None: system_histogram = system_elem.find("histogram") if system_histogram is not None: nbuckets = int(system_histogram.get("nbuckets", "0")) + overflow_count = int(system_histogram.get("overflow_count", "0")) buckets = [] + overflow_bucket = None for bucket in system_histogram.findall("bucket"): - index = int(bucket.get("index")) + index_str = bucket.get("index") count = int(bucket.get("value")) - buckets.append({"latency_us": index, "count": count}) + + # Handle overflow bucket specially + if index_str == "overflow": + overflow_bucket = {"type": "overflow", "count": count} + else: + index = int(index_str) + buckets.append({"latency_us": index, "count": count}) result["system_histogram"] = { "nbuckets": nbuckets, "buckets": buckets, - "total_samples": sum(b["count"] for b in buckets) + "total_samples": sum(b["count"] for b in buckets), + "overflow_count": overflow_count, + "overflow_bucket": overflow_bucket } # Extract per-CPU histograms - for core in timerlat.findall("core"): + for core in measurement.findall("core"): core_id = core.get("id") priority = core.get("priority") histogram = core.find("histogram") if histogram is not None: nbuckets = int(histogram.get("nbuckets", "0")) + overflow_count = int(histogram.get("overflow_count", "0")) buckets = [] + overflow_bucket = None for bucket in histogram.findall("bucket"): - index = int(bucket.get("index")) + index_str = bucket.get("index") count = int(bucket.get("value")) - buckets.append({"latency_us": index, "count": count}) + + # Handle overflow bucket specially + if index_str == "overflow": + overflow_bucket = {"type": "overflow", "count": count} + else: + index = int(index_str) + buckets.append({"latency_us": index, "count": count}) result["per_cpu_histograms"].append({ "cpu_id": core_id, "priority": priority, "nbuckets": nbuckets, "buckets": buckets, - "total_samples": sum(b["count"] for b in buckets) + "total_samples": sum(b["count"] for b in buckets), + "overflow_count": overflow_count, + "overflow_bucket": overflow_bucket }) return result
@@ -1603,7 +1626,11 @@ async def call_tool(name: str, arguments: Any) -> list[TextContent]: result += "System-Wide Histogram:\n" result += f" Total Buckets: {sys_hist['nbuckets']}\n" result += f" Total Samples: {sys_hist['total_samples']:,}\n" - result += f" Latency Range: {sys_hist['buckets'][0]['latency_us']} - {sys_hist['buckets'][-1]['latency_us']} µs\n\n" + result += f" Latency Range: {sys_hist['buckets'][0]['latency_us']} - {sys_hist['buckets'][-1]['latency_us']} µs\n" + if sys_hist['overflow_count'] > 0: + overflow_pct = (sys_hist['overflow_count'] / (sys_hist['total_samples'] + sys_hist['overflow_count'])) * 100 + result += f" ⚠️ Overflow Samples: {sys_hist['overflow_count']:,} ({overflow_pct:.3f}%) - exceeded histogram range\n" + result += "\n" # Show top 10 buckets by count top_buckets = sorted(sys_hist['buckets'], key=lambda b: b['count'], reverse=True)[:10]
@@ -1620,7 +1647,9 @@ async def call_tool(name: str, arguments: Any) -> list[TextContent]: cpu_id = cpu_hist["cpu_id"] total = cpu_hist["total_samples"] max_lat = cpu_hist["buckets"][-1]["latency_us"] if cpu_hist["buckets"] else 0 - result += f" CPU {cpu_id}: {total:>12,} samples, max latency {max_lat} µs\n" + overflow = cpu_hist["overflow_count"] + overflow_str = f", overflow: {overflow}" if overflow > 0 else "" + result += f" CPU {cpu_id}: {total:>12,} samples, max latency {max_lat} µs{overflow_str}\n" return [TextContent(type="text", text=result)]
Signature · 1 line
2.55.0
PATCH 16/23 mcp-server: Make tests generic and add Makefile integration
In reply to John KacurMake MCP server tests more robust by using dynamic test data discovery instead of hardcoded file paths. This allows tests to run with any available rteval result files.
Test improvements:
- Use glob patterns to find any rteval-*/summary.xml files
- Automatically select the most recent test data
- Add clear error messages when no test data is available
- Exit cleanly with instructions to generate test data
Makefile integration:
- Add mcp-tests target to run all MCP server tests
- Update help output to document new test target
- Add to .PHONY declarations
Tests now work with any available rteval data and provide helpful feedback when data is missing.
--- Makefile | 10 +++++++++- mcp-server/tests/test_histogram.py | 13 ++++++++++++- mcp-server/tests/test_per_cpu_stats.py | 13 ++++++++----- 3 files changed, 29 insertions(+), 7 deletions(-)
Makefile +9 -1
@@ -60,6 +60,13 @@ unit-tests: @echo "Running unit tests..." ./tests/run_tests.sh +mcp-tests: + @echo "Running MCP server tests..." + @echo "These tests require rteval result files to be present" + @echo "" + $(PYTHON) mcp-server/tests/test_histogram.py + $(PYTHON) mcp-server/tests/test_per_cpu_stats.py + tests: unit-tests test-all:
@@ -135,6 +142,7 @@ help: @echo " runit: do a short testrun locally [default]" @echo " tests: run unit tests (alias for unit-tests)" @echo " unit-tests: run unit tests" + @echo " mcp-tests: run MCP server tests (requires rteval result files)" @echo " test-all: run ALL tests including root-required tests (requires root)" @echo " e2e-tests: run end-to-end tests" @echo " regression-tests: run regression tests for measurement modules (requires root)"
@@ -157,4 +165,4 @@ tags: cleantags: rm -f tags -.PHONY: tests unit-tests test-all e2e-tests regression-tests cpuset-tests uninstall +.PHONY: tests unit-tests mcp-tests test-all e2e-tests regression-tests cpuset-tests uninstall
mcp-server/tests/test_histogram.py +12 -1
@@ -13,7 +13,18 @@ from server import extract_histogram_data, calculate_percentiles # Find test file relative to script location # From tests/ -> mcp-server/ -> rteval/ script_dir = Path(__file__).parent -test_file = script_dir.parent.parent / "rteval-20260714-1" / "summary.xml" +rteval_dir = script_dir.parent.parent + +# Find any rteval result directory with summary.xml +test_files = list(rteval_dir.glob("rteval-*/summary.xml")) +if not test_files: + print("ERROR: No rteval result files found in", rteval_dir) + print("Please run rteval to generate test data first.") + sys.exit(1) + +# Use the most recent file +test_file = sorted(test_files)[-1] +print(f"Using test file: {test_file.parent.name}/summary.xml\n") print("Testing histogram extraction...") print("=" * 60)
mcp-server/tests/test_per_cpu_stats.py +8 -5
@@ -14,15 +14,18 @@ async def main(): print("Testing get_per_cpu_stats tool") print("=" * 60) - # Find test data + # Find test data - use any available rteval result rteval_dir = Path(__file__).parent.parent.parent - test_file = rteval_dir / "rteval-20260714-1" / "summary.xml" + test_files = list(rteval_dir.glob("rteval-*/summary.xml")) - if not test_file.exists(): - print(f"✗ Test file not found: {test_file}") + if not test_files: + print(f"✗ No rteval result files found in {rteval_dir}") + print("Please run rteval to generate test data first.") sys.exit(1) - print(f"\nUsing test file: {test_file.name}\n") + # Use the most recent file + test_file = sorted(test_files)[-1] + print(f"\nUsing test file: {test_file.parent.name}/summary.xml\n") # Test 1: Basic per-CPU stats (sorted by maximum) print("1. Basic per-CPU stats sorted by maximum latency...")
Signature · 1 line
2.55.0
PATCH 17/23 mcp-server: Add load information extraction and filtering
In reply to John KacurAdd comprehensive load information analysis capabilities to the rteval MCP server. This allows users to query and filter rteval test results based on load configuration and metrics.
New functionality includes:
- extract_load_info() helper function to extract load configuration (load average, CPUs used, load generators) from rteval XML files
- get_load_info MCP tool to display load configuration for a single rteval result file
- Enhanced filter_results tool with three new filtering parameters:
- load_type: Filter by load generator (kcompile, hackbench, stressng)
- min_load_average: Minimum load average threshold
- max_load_average: Maximum load average threshold
- Enhanced batch_analysis tool to display load information for each result and aggregate load average statistics
- Updated extract_rteval_data() to include load_info section in the returned data structure
- Comprehensive documentation in README.md with:
- Complete tool usage guide showing both technical tool call format and natural language query examples for all 11 MCP tools
- Load filtering examples and use cases
- Updated feature descriptions
This enables users to query load configurations with natural language such as "Show me rteval results that used hackbench" or "Find results with load average between 1000 and 1500".
--- mcp-server/README.md | 241 ++++++++++++++++++++++++++++++++++++++++++- mcp-server/server.py | 178 ++++++++++++++++++++++++++++++++ 2 files changed, 416 insertions(+), 3 deletions(-)
mcp-server/README.md +238 -3
@@ -53,6 +53,9 @@ sudo dnf install python3-mcp python3-mcp+cli python3-lxml - **get_percentiles**: Calculate latency percentiles (P50, P95, P99, P99.9, etc.) - **get_per_cpu_stats**: Get per-CPU latency statistics and identify problematic cores +### Load Analysis +- **get_load_info**: Extract load configuration and metrics from an rteval result file + ## Usage ### Testing the Server Directly
@@ -129,6 +132,231 @@ Which CPUs have the most variable latency? mcp dev server.py ``` +## Complete Tool Usage Guide + +This section shows both the technical tool call format and natural language queries users can ask. + +### list_results + +**Tool Call:** +```python +list_results(directory=".", pattern="*.xml") +``` + +**User Asks:** +- "List all rteval results" +- "Show me rteval result files in ~/src/rteval" +- "What rteval results are in the current directory?" + +--- + +### parse_result + +**Tool Call:** +```python +parse_result(file_path="rteval-20260724-1/summary.xml") +``` + +**User Asks:** +- "Parse rteval-20260724-1" +- "Show me the results from rteval-20260724-1" +- "What's in the rteval-20260724-1 summary?" +- "Analyze rteval-20260724-1/summary.xml" + +--- + +### compare_results + +**Tool Call:** +```python +compare_results(file1="rteval-20260724-1/summary.xml", + file2="rteval-20260724-2/summary.xml") +``` + +**User Asks:** +- "Compare rteval-20260724-1 and rteval-20260724-2" +- "What's the difference between rteval-20260724-1 and rteval-20260724-2?" +- "Show me a comparison of these two rteval runs" + +--- + +### batch_analysis + +**Tool Call:** +```python +batch_analysis(directory=".", pattern="*.xml", recursive=False) +``` + +**User Asks:** +- "Analyze all rteval results" +- "Show me aggregate statistics for all rteval runs" +- "What's the average max latency across all results?" +- "Summarize all rteval results" +- "What's the range of load averages across all runs?" + +--- + +### filter_results + +**Tool Call:** +```python +filter_results(directory=".", + kernel_pattern="7.0", + is_rt=True, + date_from="2026-06-01", + date_to="2026-07-31", + min_duration_minutes=5, + max_latency_threshold=6000, + load_type="hackbench", + min_load_average=1000, + max_load_average=1500) +``` + +**User Asks:** +- "Show me rteval results with kernel 7.0" +- "Find rteval results from June 2026" +- "Show results with max latency under 6000 microseconds" +- "Filter rteval results by RT kernel" +- "Show me results that used hackbench" +- "Find runs with load average between 1000 and 1500" +- "Which results ran with kcompile load?" +- "Show me RT kernel runs with stress-ng that had low latency" + +--- + +### find_best_worst + +**Tool Call:** +```python +find_best_worst(directory=".", metric="maximum", count=5) +``` + +**User Asks:** +- "Find the 3 best and worst rteval runs by maximum latency" +- "Which rteval runs had the lowest mean latency?" +- "Show me the 5 best and worst runs" +- "What were the top performing runs by median latency?" + +--- + +### compare_to_baseline + +**Tool Call:** +```python +compare_to_baseline(baseline_file="rteval-20260605-1/summary.xml", + directory=".", + threshold_percent=10) +``` + +**User Asks:** +- "Compare all rteval results to rteval-20260605-1/summary.xml" +- "Check for regressions against the baseline" +- "Which runs regressed compared to rteval-20260605-1?" +- "Are there any performance regressions with 5% threshold?" + +--- + +### list_logs + +**Tool Call:** +```python +list_logs(result_dir="rteval-20260724-1") +``` + +**User Asks:** +- "List logs in rteval-20260724-1" +- "What log files are available for rteval-20260724-1?" +- "Show me the logs from this run" + +--- + +### read_log + +**Tool Call:** +```python +read_log(log_path="rteval-20260724-1/logs/timerlat.stdout", + tail=100, + grep="error") +``` + +**User Asks:** +- "Show me the last 100 lines of the timerlat log" +- "Read the timerlat stdout from rteval-20260724-1" +- "Search for 'error' in the timerlat log" +- "Show me the beginning of the kcompile log" + +--- + +### extract_histogram + +**Tool Call:** +```python +extract_histogram(file_path="rteval-20260714-1/summary.xml", + include_per_cpu=True) +``` + +**User Asks:** +- "Extract histogram data from rteval-20260714-1" +- "Show me the latency distribution for rteval-20260714-1" +- "What does the histogram look like?" +- "Show me per-CPU histogram data" + +--- + +### get_percentiles + +**Tool Call:** +```python +get_percentiles(file_path="rteval-20260714-1/summary.xml", + percentiles=[50, 90, 95, 99, 99.9, 99.99], + per_cpu=False) +``` + +**User Asks:** +- "Calculate P99 and P99.9 percentiles for rteval-20260714-1" +- "Show me percentiles for each CPU in rteval-20260714-1" +- "What percentage of samples had latency under 10 microseconds?" +- "What's the P95 latency?" +- "Show me per-CPU percentiles" + +--- + +### get_per_cpu_stats + +**Tool Call:** +```python +get_per_cpu_stats(file_path="rteval-20260714-1/summary.xml", + sort_by="maximum", + show_top_n=5, + highlight_threshold=1000) +``` + +**User Asks:** +- "Show me per-CPU statistics for rteval-20260714-1" +- "Which CPUs had the worst maximum latency?" +- "Show the top 5 CPUs sorted by mean latency" +- "Highlight CPUs with max latency above 1000 microseconds" +- "Which CPUs have the most variable latency?" +- "Which CPU had the worst latency spike?" + +--- + +### get_load_info + +**Tool Call:** +```python +get_load_info(file_path="rteval-20260724-1/summary.xml") +``` + +**User Asks:** +- "What loads were running in rteval-20260724-1?" +- "Show me the load configuration for rteval-20260724-1" +- "What was the load average in this run?" +- "What load generators were used?" +- "Show me the load setup for this test" + +--- + ## Features in Detail ### filter_results
@@ -138,8 +366,14 @@ Filter rteval results by multiple criteria: - **date_from/date_to**: Date range filter (YYYY-MM-DD format) - **min_duration_minutes**: Minimum test duration - **max_latency_threshold**: Maximum acceptable latency in µs +- **load_type**: Filter by load generator type (e.g., "kcompile", "hackbench", "stressng") +- **min_load_average**: Minimum load average threshold +- **max_load_average**: Maximum load average threshold -Example: Find all RT kernel runs from June 2026 with max latency under 4000 µs +Examples: +- Find all RT kernel runs from June 2026 with max latency under 4000 µs +- Find runs with hackbench load and load average between 1000 and 1500 +- Show runs with stress-ng that had low latency on RT kernels ### find_best_worst Identify optimal and poorest performing runs:
@@ -164,9 +398,10 @@ Aggregate statistics across multiple runs: - Total files analyzed - Date range coverage - Min/max/average latencies across all runs -- Individual result summaries +- Min/max/average load averages across all runs +- Individual result summaries with load information -Useful for understanding performance trends over time. +Useful for understanding performance trends over time and correlating load with latency. ### extract_histogram Extract raw histogram data from rteval results:
mcp-server/server.py +178 -0
244 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 18/23 mcp-server: Rename server from rteval to rteval-mcp
In reply to John KacurRename the MCP server name from "rteval" to "rteval-mcp" to avoid naming conflicts and follow MCP server naming conventions.
This updates:
- Server name in .mcp.json configuration
- Plugin metadata in manifest.json and plugin.json
- Tool prefix documentation in README (from mcp__plugin_rteval-mcp_rteval__ to mcp__rteval-mcp__)
--- .mcp.json | 7 +++---- mcp-server/.claude-plugin/plugin.json | 2 +- mcp-server/README.md | 2 +- mcp-server/manifest.json | 2 +- 4 files changed, 6 insertions(+), 7 deletions(-)
.mcp.json +3 -4
@@ -1,11 +1,10 @@ { "mcpServers": { - "rteval": { + "rteval-mcp": { "command": "python3", "args": [ "/home/jkacur/src/rteval/mcp-server/server.py" - ], - "env": {} + ] } } -} +}
mcp-server/.claude-plugin/plugin.json +1 -1
@@ -9,7 +9,7 @@ "license": "GPL-2.0", "keywords": ["rteval", "real-time", "latency", "testing", "timerlat", "cyclictest"], "mcpServers": { - "rteval": { + "rteval-mcp": { "command": "python3", "args": [ "/home/jkacur/src/rteval/mcp-server/server.py"
mcp-server/README.md +1 -1
@@ -71,7 +71,7 @@ The server is configured as a Claude Code plugin. After installation: 1. The server is registered in `~/.claude/plugins/installed_plugins.json` 2. Enabled in `~/.claude/settings.json` 3. Restart Claude Code to load the plugin -4. Tools will be available with the prefix: `mcp__plugin_rteval-mcp_rteval__` +4. Tools will be available with the prefix: `mcp__rteval-mcp__` Example queries: ```
mcp-server/manifest.json +1 -1
@@ -1,6 +1,6 @@ { "manifest_version": "0.2", - "name": "rteval", + "name": "rteval-mcp", "version": "0.1.0", "description": "rteval test results analysis - query and compare real-time evaluation test results", "author": {
Signature · 1 line
2.55.0
PATCH 19/23 mcp-server: Add GPL-2.0-or-later license headers to all files
In reply to John KacurAdd SPDX license identifiers and copyright notices to ensure GPL compatibility with the rest of rteval. The MCP server code is part of rteval and must be licensed under GPL-2.0-or-later to be compatible with the main project.
Changes:
- Add GPL-2.0-or-later SPDX identifier to all Python files
- Add copyright notice: Copyright 2026 John Kacur <[email protected]>
- Add UTF-8 encoding declaration to all Python files
- Add License section to README.md with GPL boilerplate
All files now follow the same licensing pattern as the rest of rteval, with proper SPDX identifiers in the header comments.
--- mcp-server/DEPENDENCIES.md | 8 ++++++++ mcp-server/README.md | 11 +++++++++++ mcp-server/server.py | 7 +++++++ mcp-server/tests/test_histogram.py | 7 +++++++ mcp-server/tests/test_mcp_integration.py | 7 +++++++ mcp-server/tests/test_new_tools.py | 7 +++++++ mcp-server/tests/test_per_cpu_stats.py | 7 +++++++ 7 files changed, 54 insertions(+)
mcp-server/DEPENDENCIES.md +8 -0
@@ -1,3 +1,11 @@ +<!-- +SPDX-License-Identifier: GPL-2.0-or-later + +rteval MCP Server - Dependencies + +Copyright 2026 John Kacur <[email protected]> +--> + # Dependencies ## Fedora Packages
mcp-server/README.md +11 -0
@@ -2,6 +2,17 @@ An MCP (Model Context Protocol) server for querying and analyzing rteval test results. +## License + +SPDX-License-Identifier: GPL-2.0-or-later + +Copyright 2026 John Kacur <[email protected]> + +This program is free software; you can redistribute it and/or modify +it under the terms of the GNU General Public License as published by +the Free Software Foundation; either version 2 of the License, or +(at your option) any later version. + ## Purpose This MCP server provides tools to:
mcp-server/server.py +7 -0
@@ -1,4 +1,11 @@ #!/usr/bin/env python3 +# -*- coding: utf-8 -*- +# SPDX-License-Identifier: GPL-2.0-or-later +# +# rteval MCP Server - tools for analyzing rteval results +# +# Copyright 2026 John Kacur <[email protected]> +# """ rteval MCP Server
mcp-server/tests/test_histogram.py +7 -0
@@ -1,4 +1,11 @@ #!/usr/bin/env python3 +# -*- coding: utf-8 -*- +# SPDX-License-Identifier: GPL-2.0-or-later +# +# rteval MCP Server tests - histogram and percentile tools +# +# Copyright 2026 John Kacur <[email protected]> +# """ Test script for histogram extraction and percentile calculation. """
mcp-server/tests/test_mcp_integration.py +7 -0
@@ -1,4 +1,11 @@ #!/usr/bin/env python3 +# -*- coding: utf-8 -*- +# SPDX-License-Identifier: GPL-2.0-or-later +# +# rteval MCP Server tests - integration tests +# +# Copyright 2026 John Kacur <[email protected]> +# """Integration tests for rteval MCP server. Tests the MCP server tools through the actual MCP interface,
mcp-server/tests/test_new_tools.py +7 -0
@@ -1,4 +1,11 @@ #!/usr/bin/env python3 +# -*- coding: utf-8 -*- +# SPDX-License-Identifier: GPL-2.0-or-later +# +# rteval MCP Server tests - query/filter tools +# +# Copyright 2026 John Kacur <[email protected]> +# """Test the new query/filter tools.""" import sys
mcp-server/tests/test_per_cpu_stats.py +7 -0
@@ -1,4 +1,11 @@ #!/usr/bin/env python3 +# -*- coding: utf-8 -*- +# SPDX-License-Identifier: GPL-2.0-or-later +# +# rteval MCP Server tests - per-CPU statistics +# +# Copyright 2026 John Kacur <[email protected]> +# """Test the get_per_cpu_stats tool.""" import sys
Signature · 1 line
2.55.0
PATCH 20/23 mcp-server: Add Makefile for running tests
In reply to John KacurAdd a local Makefile in the mcp-server directory to simplify running tests. This keeps the MCP server tests isolated since the MCP server may be an optional package that not all users will have installed.
The Makefile provides:
- make test: Run all tests
- make test-new-tools: Query/filter tools tests
- make test-integration: MCP integration tests
- make test-histogram: Histogram extraction tests
- make test-per-cpu: Per-CPU statistics tests
- make help: Show available targets
- make clean: Clean Python cache files
Updated README.md to document the Makefile-based test workflow, replacing the outdated reference to test_tools.py which no longer exists.
The Makefile can be invoked from the mcp-server directory or from the project root using: make -C mcp-server test
--- mcp-server/Makefile | 51 ++++++++++++++++++++++++++++++++++++++++++++ mcp-server/README.md | 21 ++++++++++++++++-- 2 files changed, 70 insertions(+), 2 deletions(-) create mode 100644 mcp-server/Makefile
mcp-server/Makefile new file +51 -0
@@ -0,0 +1,51 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Makefile for rteval MCP Server tests +# +# Copyright 2026 John Kacur <[email protected]> + +PYTHON := python3 +TEST_DIR := tests + +.PHONY: all help test test-new-tools test-integration test-histogram test-per-cpu clean + +all: test + +help: + @echo "rteval MCP Server - Test targets:" + @echo "" + @echo " make - Run all tests (default)" + @echo " make test - Run all tests" + @echo " make test-new-tools - Run query/filter tools tests" + @echo " make test-integration - Run MCP integration tests" + @echo " make test-histogram - Run histogram extraction tests" + @echo " make test-per-cpu - Run per-CPU statistics tests" + @echo " make clean - Clean Python cache files" + @echo "" + +test: test-new-tools test-integration test-histogram test-per-cpu + @echo "" + @echo "All tests completed successfully" + +test-new-tools: + @echo "Running query/filter tools tests..." + @cd $(TEST_DIR) && $(PYTHON) test_new_tools.py + +test-integration: + @echo "Running MCP integration tests..." + @cd $(TEST_DIR) && $(PYTHON) test_mcp_integration.py + +test-histogram: + @echo "Running histogram extraction tests..." + @cd $(TEST_DIR) && $(PYTHON) test_histogram.py + +test-per-cpu: + @echo "Running per-CPU statistics tests..." + @cd $(TEST_DIR) && $(PYTHON) test_per_cpu_stats.py + +clean: + @echo "Cleaning Python cache files..." + @find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true + @find . -type f -name '*.pyc' -delete + @find . -type f -name '*.pyo' -delete + @echo "Clean completed"
mcp-server/README.md +19 -2
@@ -69,10 +69,27 @@ sudo dnf install python3-mcp python3-mcp+cli python3-lxml ## Usage -### Testing the Server Directly +### Testing the Server + +The MCP server includes a test suite to verify functionality: ```bash -python3 test_tools.py +cd mcp-server + +# Run all tests +make test + +# Run individual test suites +make test-new-tools # Query/filter tools +make test-integration # MCP integration tests +make test-histogram # Histogram extraction +make test-per-cpu # Per-CPU statistics + +# Show available targets +make help + +# Clean Python cache files +make clean ``` ### Using with Claude Code
Signature · 1 line
2.55.0
PATCH 21/23 mcp-server: Add demo guide with example queries
In reply to John KacurAdd DEMO.md to provide users with example queries demonstrating the MCP server capabilities. The guide includes:
- Prerequisites and test data generation instructions
- Basic operations (listing, parsing, comparing results)
- Query and filtering examples (kernel, date, latency, load)
- Best/worst analysis queries
- Histogram and percentile analysis examples
- Per-CPU statistics and analysis queries
- Load configuration analysis
- Batch analysis and baseline comparison
- Log analysis examples
- Advanced multi-criteria queries
- Tips for effective queries and troubleshooting
Each section includes natural language queries users can ask Claude Code along with expected results, making it easy for users to explore the MCP server functionality.
The demo guide complements README.md by focusing on practical usage examples rather than technical documentation.
--- mcp-server/DEMO.md | 462 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 462 insertions(+) create mode 100644 mcp-server/DEMO.md
mcp-server/DEMO.md new file +462 -0
@@ -0,0 +1,462 @@ +# rteval MCP Server Demo Guide + +This document provides example queries to demonstrate the capabilities of the rteval MCP server when used with Claude Code. + +## License + +SPDX-License-Identifier: GPL-2.0-or-later + +Copyright 2026 John Kacur <[email protected]> + +This program is free software; you can redistribute it and/or modify +it under the terms of the GNU General Public License as published by +the Free Software Foundation; either version 2 of the License, or +(at your option) any later version. + +## Prerequisites + +Before running the demo, ensure you have rteval result files available: + +```bash +# Generate some test data if needed +cd ~/src/rteval +sudo ./rteval-cmd --duration=5m + +# Or use existing results in the rteval directory +ls -d rteval-*/ +``` + +## Basic Operations + +### Discovering Results + +**Query:** "List all rteval results in the current directory" + +**Expected:** Shows all rteval-YYYYMMDD-N directories with summary.xml files + +--- + +**Query:** "What rteval results are available?" + +**Expected:** Same as above - lists available result directories + +--- + +### Parsing Individual Results + +**Query:** "Parse the latest rteval result" + +**Expected:** Shows comprehensive summary including: +- System info (kernel version, architecture, CPU count) +- Test duration and date +- Maximum, mean, and median latency +- Load information + +--- + +**Query:** "Show me the results from rteval-20260714-1" + +**Expected:** Detailed metrics for that specific run + +--- + +### Comparing Two Results + +**Query:** "Compare rteval-20260714-1 and rteval-20260715-1" + +**Expected:** Side-by-side comparison showing: +- Kernel versions +- Latency differences (max, mean, median) +- Load configuration differences +- Performance improvement/regression percentages + +--- + +## Query and Filtering + +### Kernel-Based Filtering + +**Query:** "Show me rteval results with kernel 7.0" + +**Expected:** Filtered list showing only results running kernel 7.0.x + +--- + +**Query:** "Find rteval results running RT kernels" + +**Expected:** Results filtered to show only RT (PREEMPT_RT) kernels + +--- + +### Date Range Filtering + +**Query:** "Show me rteval results from June 2026" + +**Expected:** Results filtered by date range + +--- + +**Query:** "Find rteval results from the last week" + +**Expected:** Recent results within the specified timeframe + +--- + +### Latency Threshold Filtering + +**Query:** "Show results with max latency under 5000 microseconds" + +**Expected:** Only results where maximum latency is below the threshold + +--- + +**Query:** "Find rteval runs with excellent latency (max < 3000 µs)" + +**Expected:** High-performing results meeting the latency criteria + +--- + +### Load Generator Filtering + +**Query:** "Show me results that used hackbench" + +**Expected:** Results filtered by load generator type + +--- + +**Query:** "Find runs with kcompile load and high load average" + +**Expected:** Results using kcompile with specified load characteristics + +--- + +## Best/Worst Analysis + +### Finding Best Runs + +**Query:** "Find the 5 best rteval runs by maximum latency" + +**Expected:** Top 5 runs ranked by lowest maximum latency + +--- + +**Query:** "Which rteval runs had the lowest mean latency?" + +**Expected:** Ranked list showing best average performance + +--- + +**Query:** "Show me the best runs by median latency" + +**Expected:** Rankings based on median latency metric + +--- + +### Finding Worst Runs + +**Query:** "Find the 3 worst rteval runs" + +**Expected:** Shows the runs with highest maximum latency + +--- + +**Query:** "Which runs had the poorest performance?" + +**Expected:** Identifies problematic runs with high latencies + +--- + +## Histogram and Percentile Analysis + +### Extracting Histogram Data + +**Query:** "Extract histogram data from rteval-20260714-1" + +**Expected:** Shows: +- Total sample count +- Latency range (min to max) +- Top latency buckets with counts and percentages +- Per-CPU histogram summaries + +--- + +**Query:** "Show me the latency distribution for the latest rteval run" + +**Expected:** Histogram showing how latency values are distributed + +--- + +### Percentile Calculations + +**Query:** "Calculate P99 and P99.9 percentiles for rteval-20260714-1" + +**Expected:** Shows percentile values like: +- P50 (median) +- P90 +- P95 +- P99 +- P99.9 +- P99.99 + +--- + +**Query:** "What's the P95 latency for the latest run?" + +**Expected:** Specific percentile value showing that 95% of samples were below this latency + +--- + +**Query:** "Show me per-CPU percentiles for rteval-20260714-1" + +**Expected:** Percentile breakdown for each individual CPU + +--- + +**Query:** "What percentage of samples had latency under 10 microseconds?" + +**Expected:** Uses histogram to calculate the cumulative percentage + +--- + +## Per-CPU Analysis + +### Basic Per-CPU Statistics + +**Query:** "Show me per-CPU statistics for rteval-20260714-1" + +**Expected:** Table showing for each CPU: +- Sample count +- Min, max, mean, median latency +- Standard deviation + +--- + +**Query:** "Which CPUs had the worst maximum latency?" + +**Expected:** CPUs sorted by maximum latency (worst first) + +--- + +### Identifying Problem CPUs + +**Query:** "Show the top 5 CPUs sorted by mean latency" + +**Expected:** Table limited to 5 worst CPUs by average latency + +--- + +**Query:** "Highlight CPUs with max latency above 5000 microseconds" + +**Expected:** Table with ⚠️ warning symbols next to CPUs exceeding threshold + +--- + +**Query:** "Which CPUs have the most variable latency?" + +**Expected:** CPUs sorted by standard deviation (highest first) + +--- + +**Query:** "Which CPU had the worst latency spike?" + +**Expected:** Single CPU with the absolute highest maximum latency + +--- + +### Consistency Analysis + +**Query:** "Show me the most consistent CPUs" + +**Expected:** CPUs sorted by standard deviation (lowest first = most consistent) + +--- + +**Query:** "Are any CPUs consistently slow?" + +**Expected:** Analysis of mean latency to find systematically poor performers + +--- + +## Load Analysis + +### Load Configuration + +**Query:** "What loads were running in rteval-20260714-1?" + +**Expected:** Shows: +- Load generator types (hackbench, kcompile, stressng) +- Load configuration parameters +- Load average (1-min, 5-min, 15-min) + +--- + +**Query:** "Show me the load setup for the latest test" + +**Expected:** Complete load configuration and metrics + +--- + +**Query:** "What was the load average in this run?" + +**Expected:** System load averages during the test + +--- + +## Batch Analysis + +### Aggregate Statistics + +**Query:** "Analyze all rteval results" + +**Expected:** Aggregate statistics showing: +- Total files analyzed +- Date range coverage +- Min/max/average latencies across all runs +- Min/max/average load averages +- Individual result summaries + +--- + +**Query:** "What's the average max latency across all runs?" + +**Expected:** Single aggregate metric across all results + +--- + +**Query:** "Show me the range of load averages across all runs" + +**Expected:** Min and max load averages from all results + +--- + +### Baseline Comparison + +**Query:** "Compare all rteval results to rteval-20260605-1/summary.xml" + +**Expected:** Shows: +- Baseline metrics +- Regressions (results exceeding threshold) +- Passes (results within threshold) +- Detailed percentage changes for all comparisons + +--- + +**Query:** "Check for regressions against the baseline with 5% threshold" + +**Expected:** Regression detection with custom threshold + +--- + +**Query:** "Which runs regressed compared to rteval-20260605-1?" + +**Expected:** List of runs that performed worse than baseline + +--- + +## Log Analysis + +### Listing Logs + +**Query:** "List logs in rteval-20260714-1" + +**Expected:** Shows available log files: +- timerlat.stdout +- hackbench.stdout +- kcompile.stdout +- dmesg +- etc. + +--- + +**Query:** "What log files are available for the latest run?" + +**Expected:** Complete list of log files in the result directory + +--- + +### Reading Logs + +**Query:** "Show me the last 100 lines of the timerlat log" + +**Expected:** Tail of the timerlat stdout log + +--- + +**Query:** "Search for 'error' in the timerlat log" + +**Expected:** Filtered log showing only lines containing 'error' + +--- + +**Query:** "Show me the beginning of the kcompile log" + +**Expected:** First lines (head) of the kcompile log + +--- + +## Advanced Queries + +### Multi-Criteria Filtering + +**Query:** "Show me RT kernel runs from June 2026 with max latency under 4000 µs" + +**Expected:** Results matching all three criteria: +- RT kernel +- Date in June 2026 +- Max latency < 4000 µs + +--- + +**Query:** "Find runs with hackbench load and load average between 1000 and 1500" + +**Expected:** Results filtered by load type and load average range + +--- + +### Comparative Analysis + +**Query:** "How does kernel 7.0 compare to kernel 6.17 in terms of latency?" + +**Expected:** Filtering and comparison showing performance differences between kernel versions + +--- + +**Query:** "Show the latency trend over time" + +**Expected:** Batch analysis ordered by date showing temporal patterns + +--- + +## Tips for Effective Queries + +1. **Be specific with file paths:** Use actual rteval directory names from your system +2. **Combine criteria:** Most tools support multiple filters for precise analysis +3. **Use natural language:** The MCP server understands conversational queries +4. **Ask follow-ups:** Build on previous queries to dive deeper into results +5. **Verify thresholds:** When filtering by latency, make sure thresholds match your RT requirements + +## Expected Data Requirements + +For the best demo experience, have: +- At least 5-10 rteval result directories +- Results from different kernel versions (preferably mix of RT and non-RT) +- Results spanning different dates +- Variety of load generators (hackbench, kcompile, stressng) +- Some high-latency and some low-latency results for comparison + +## Troubleshooting Demo Issues + +If queries don't work as expected: + +1. **No results found:** Check that rteval result directories exist with summary.xml files +2. **Empty filters:** Adjust filter criteria to match your actual test data +3. **Tool not found:** Verify the MCP server is properly installed and enabled in Claude Code +4. **Permission errors:** Ensure rteval results are readable by your user + +## Next Steps + +After completing the demo: +- Review the full tool documentation in README.md +- Run `make test` to verify the MCP server installation +- Create baseline results for your specific RT requirements +- Set up automated regression testing using the baseline comparison tools
Signature · 1 line
2.55.0
PATCH 22/23 mcp-server: Restructure as Python package with dynamic paths
In reply to John KacurRestructure the MCP server into a standard Python package to enable both development and system installation scenarios.
Directory and file changes:
- Rename mcp-server/ to rteval_mcp/ (Python package naming convention)
- Rename server.py to rteval_server.py (avoid naming conflicts)
- Add __init__.py with package metadata
- Add __main__.py to enable 'python3 -m rteval_mcp' execution
- Create rteval-mcp-server wrapper script for command-line execution
- Update all test imports from 'server' to 'rteval_server'
Configuration files now use relative paths instead of hardcoded paths:
- manifest.json: ./rteval_mcp/rteval-mcp-server
- .claude-plugin/plugin.json: ./rteval_mcp/rteval-mcp-server
The wrapper script dynamically calculates paths:
- Finds its own location using BASH_SOURCE
- Sets PYTHONPATH only when running from git (detects .git directory)
- For system installations, relies on standard site-packages location
This approach supports both development from git (./rteval-cmd pattern) and future system packaging without path conflicts.
---
.../.claude-plugin/plugin.json | 6 ++--
{mcp-server => rteval_mcp}/DEMO.md | 0
{mcp-server => rteval_mcp}/DEPENDENCIES.md | 0
{mcp-server => rteval_mcp}/Makefile | 0
{mcp-server => rteval_mcp}/README.md | 4 +--
rteval_mcp/__init__.py | 13 ++++++++
rteval_mcp/__main__.py | 33 +++++++++++++++++++
{mcp-server => rteval_mcp}/manifest.json | 8 ++---
rteval_mcp/rteval-mcp-server | 21 ++++++++++++
.../server.py => rteval_mcp/rteval_server.py | 0
.../tests/test_histogram.py | 4 +--
.../tests/test_mcp_integration.py | 4 +--
.../tests/test_new_tools.py | 4 +--
.../tests/test_per_cpu_stats.py | 4 +--
14 files changed, 82 insertions(+), 19 deletions(-)
rename {mcp-server => rteval_mcp}/.claude-plugin/plugin.json (79%)
rename {mcp-server => rteval_mcp}/DEMO.md (100%)
rename {mcp-server => rteval_mcp}/DEPENDENCIES.md (100%)
rename {mcp-server => rteval_mcp}/Makefile (100%)
rename {mcp-server => rteval_mcp}/README.md (99%)
create mode 100644 rteval_mcp/__init__.py
create mode 100644 rteval_mcp/__main__.py
rename {mcp-server => rteval_mcp}/manifest.json (70%)
create mode 100755 rteval_mcp/rteval-mcp-server
rename mcp-server/server.py => rteval_mcp/rteval_server.py (100%)
rename {mcp-server => rteval_mcp}/tests/test_histogram.py (95%)
rename {mcp-server => rteval_mcp}/tests/test_mcp_integration.py (98%)
rename {mcp-server => rteval_mcp}/tests/test_new_tools.py (97%)
rename {mcp-server => rteval_mcp}/tests/test_per_cpu_stats.py (97%)rteval_mcp/.claude-plugin/plugin.json renamed from mcp-server/.claude-plugin/plugin.json +2 -4
@@ -10,10 +10,8 @@ "keywords": ["rteval", "real-time", "latency", "testing", "timerlat", "cyclictest"], "mcpServers": { "rteval-mcp": { - "command": "python3", - "args": [ - "/home/jkacur/src/rteval/mcp-server/server.py" - ], + "command": "./rteval_mcp/rteval-mcp-server", + "args": [], "env": {} } }
rteval_mcp/DEMO.md renamed from mcp-server/DEMO.md +0 -0
rteval_mcp/DEPENDENCIES.md renamed from mcp-server/DEPENDENCIES.md +0 -0
rteval_mcp/Makefile renamed from mcp-server/Makefile +0 -0
rteval_mcp/README.md renamed from mcp-server/README.md +2 -2
@@ -74,7 +74,7 @@ sudo dnf install python3-mcp python3-mcp+cli python3-lxml The MCP server includes a test suite to verify functionality: ```bash -cd mcp-server +cd rteval_mcp # Run all tests make test
@@ -157,7 +157,7 @@ Which CPUs have the most variable latency? ### Using with MCP Inspector ```bash -mcp dev server.py +mcp dev rteval-mcp-server ``` ## Complete Tool Usage Guide
rteval_mcp/__init__.py new file +13 -0
@@ -0,0 +1,13 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +# SPDX-License-Identifier: GPL-2.0-or-later +# +# rteval MCP Server package +# +# Copyright 2026 John Kacur <[email protected]> +# +"""rteval MCP Server - tools for analyzing rteval results.""" + +__version__ = "0.1.0" +__author__ = "John Kacur" +__email__ = "[email protected]"
rteval_mcp/__main__.py new file +33 -0
@@ -0,0 +1,33 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +# SPDX-License-Identifier: GPL-2.0-or-later +# +# rteval MCP Server entry point +# +# Copyright 2026 John Kacur <[email protected]> +# +"""Entry point for running rteval MCP server as a module.""" + +import sys +import asyncio +from pathlib import Path + +# Add package directory to path for development mode +package_dir = Path(__file__).parent +sys.path.insert(0, str(package_dir)) + +from rteval_server import app, stdio_server + + +async def main(): + """Run the MCP server.""" + async with stdio_server() as (read_stream, write_stream): + await app.run( + read_stream, + write_stream, + app.create_initialization_options() + ) + + +if __name__ == "__main__": + asyncio.run(main())
rteval_mcp/manifest.json renamed from mcp-server/manifest.json +3 -5
@@ -9,12 +9,10 @@ }, "server": { "type": "python", - "entry_point": "server.py", + "entry_point": "rteval-mcp-server", "mcp_config": { - "command": "python3", - "args": [ - "/home/jkacur/src/rteval/mcp-server/server.py" - ] + "command": "./rteval_mcp/rteval-mcp-server", + "args": [] } } }
rteval_mcp/rteval-mcp-server new file +21 -0
@@ -0,0 +1,21 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# +# rteval MCP Server wrapper script +# +# Copyright 2026 John Kacur <[email protected]> +# +# This wrapper executes the rteval MCP server as a Python module. +# It works for both development (local git) and installed (system) versions. + +# Find the directory containing this script +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +# Parent directory is where Python should search for rteval_mcp module +PARENT_DIR="$(dirname "$SCRIPT_DIR")" + +# Only set PYTHONPATH if running from a git development tree +if [ -d "${PARENT_DIR}/.git" ]; then + export PYTHONPATH="${PARENT_DIR}:${PYTHONPATH}" +fi + +exec python3 -m rteval_mcp "$@"
rteval_mcp/rteval_server.py renamed from mcp-server/server.py +0 -0
rteval_mcp/tests/test_histogram.py renamed from mcp-server/tests/test_histogram.py +2 -2
@@ -13,9 +13,9 @@ Test script for histogram extraction and percentile calculation. import sys from pathlib import Path -# Add parent directory to path to import server module +# Add parent directory to path to import rteval_server module sys.path.insert(0, str(Path(__file__).parent.parent)) -from server import extract_histogram_data, calculate_percentiles +from rteval_server import extract_histogram_data, calculate_percentiles # Find test file relative to script location # From tests/ -> mcp-server/ -> rteval/
rteval_mcp/tests/test_mcp_integration.py renamed from mcp-server/tests/test_mcp_integration.py +2 -2
@@ -16,9 +16,9 @@ import sys import asyncio from pathlib import Path -# Add parent directory to path to import server module +# Add parent directory to path to import rteval_server module sys.path.insert(0, str(Path(__file__).parent.parent)) -from server import call_tool, list_tools +from rteval_server import call_tool, list_tools def find_test_data():
rteval_mcp/tests/test_new_tools.py renamed from mcp-server/tests/test_new_tools.py +2 -2
@@ -12,9 +12,9 @@ import sys import asyncio from pathlib import Path -# Add parent directory to path to import server module +# Add parent directory to path to import rteval_server module sys.path.insert(0, str(Path(__file__).parent.parent)) -from server import extract_rteval_data +from rteval_server import extract_rteval_data async def test_find_best_worst():
rteval_mcp/tests/test_per_cpu_stats.py renamed from mcp-server/tests/test_per_cpu_stats.py +2 -2
@@ -12,9 +12,9 @@ import sys import asyncio from pathlib import Path -# Add parent directory to path to import server module +# Add parent directory to path to import rteval_server module sys.path.insert(0, str(Path(__file__).parent.parent)) -from server import call_tool +from rteval_server import call_tool async def main():
Signature · 1 line
2.55.0
PATCH 23/23 mcp-server: Use relative path in .mcp.json and add example for installation
In reply to John KacurChange .mcp.json to use a relative path (./rteval_mcp/rteval-mcp-server) instead of an absolute path. This allows developers to work from any git checkout location without hardcoded paths.
Add mcp.json.example for packagers and system installations. This file documents how to configure the MCP server when rteval-mcp-server is installed to a system path like /usr/bin. Packagers can install this example to /usr/share/doc/rteval/ or similar location.
--- .mcp.json | 5 +---- mcp.json.example | 11 +++++++++++ 2 files changed, 12 insertions(+), 4 deletions(-) create mode 100644 mcp.json.example
.mcp.json +1 -4
@@ -1,10 +1,7 @@ { "mcpServers": { "rteval-mcp": { - "command": "python3", - "args": [ - "/home/jkacur/src/rteval/mcp-server/server.py" - ] + "command": "./rteval_mcp/rteval-mcp-server" } } }
mcp.json.example new file +11 -0
@@ -0,0 +1,11 @@ +{ + "_comment": "Example MCP server configuration for rteval-mcp", + "_installation": "When rteval is installed system-wide, this file should be copied to ~/.config/mcp/mcp.json or merged with existing MCP configuration", + "_usage": "The rteval-mcp-server command will be installed to /usr/bin or a similar system path", + + "mcpServers": { + "rteval-mcp": { + "command": "rteval-mcp-server" + } + } +}
Signature · 1 line
2.55.0