Skip to content

Feat/multi gpu support#351

Merged
DanteNiewenhuis merged 16 commits into
atlarge-research:masterfrom
noleu:feat/multi-gpu-support
Jul 15, 2025
Merged

Feat/multi gpu support#351
DanteNiewenhuis merged 16 commits into
atlarge-research:masterfrom
noleu:feat/multi-gpu-support

Conversation

@noleu

@noleu noleu commented Jun 24, 2025

Copy link
Copy Markdown
Contributor

Summary

This PR introduces comprehensive multi-GPU support to the OpenDC datacenter simulation platform, enabling simulation of modern GPU-accelerated workloads with arbitrary numbers of GPUs per node. The implementation extends the telemetry export system to capture detailed per-GPU metrics and refactors the GPU statistics reporting mechanism for improved performance analysis.

Implementation Notes ⚒️

  • Dynamic GPU Export Columns: Extended the Parquet export functionality with a new withGpuColumns function in ComputeExportConfig that dynamically generates export columns based on the number of GPUs present, replacing hardcoded GPU column definitions
  • Aggregated GPU Statistics: Refactored the GPU statistics reporting mechanism to provide a single, aggregated view instead of individual GPU statistics lists, simplifying downstream data processing and analysis
  • Naming Convention Updates: Renamed MaxMinFairnessStrategy to MaxMinFairnessPolicy across factory and distributor components for better clarity and consistency
  • Test Code Cleanup: Removed obsolete commented-out assertions in GPU tests to improve code maintainability
  • Enhanced Telemetry Granularity: The new gpuColumns function generates detailed export columns based on GPU index, enabling comprehensive performance analysis of multi-GPU environments

External Dependencies 🍀

  • N/A

Breaking API Changes ⚠️

  • GPU Metrics Structure: The GPU metrics reporting API now returns aggregated data instead of individual GPU lists - downstream analysis code may need adjustments to handle the new data structure

noleu added 13 commits June 24, 2025 18:59
Extends the Parquet export functionality to support an arbitrary number of GPUs.
This change introduces a new function `withGpuColumns` to `ComputeExportConfig`
that dynamically generates the necessary export columns based on the number of GPUs.
The existing hardcoded GPU columns are replaced by a new `gpuColumns` function
that generates a set of export columns based on the index of the GPU.

This enhancement allows users to collect detailed telemetry data for each GPU
present in the simulated environment, improving the accuracy and granularity
of performance analysis.
Refactors the GPU statistics reporting mechanism to provide a single, aggregated view instead of a list of individual statistics.
This change simplifies data access and reduces complexity in downstream processing, allowing for clearer and more efficient analysis of GPU usage.
The task export columns are updated to reflect this change.
@noleu noleu force-pushed the feat/multi-gpu-support branch from 6bfe654 to bb9ec42 Compare June 24, 2025 17:00
@codecov

codecov Bot commented Jun 24, 2025

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 82.93651% with 43 lines in your changes missing coverage. Please review.

Project coverage is 62.62%. Comparing base (0203254) to head (df6829e).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
...ava/org/opendc/simulator/compute/power/SimPsu.java 52.45% 21 Missing and 8 partials ⚠️
...opendc/simulator/engine/graph/FlowDistributor.java 79.41% 5 Missing and 2 partials ⚠️
...ndc/simulator/compute/workload/VirtualMachine.java 88.00% 2 Missing and 1 partial ⚠️
...mulator/telemetry/parquet/DfltTaskExportColumns.kt 84.61% 2 Missing ⚠️
...g/opendc/experiments/base/runner/ScenarioRunner.kt 81.81% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master     #351      +/-   ##
============================================
+ Coverage     62.54%   62.62%   +0.07%     
  Complexity       51       51              
============================================
  Files           354      354              
  Lines         12223    12256      +33     
  Branches       1250     1236      -14     
============================================
+ Hits           7645     7675      +30     
- Misses         4132     4146      +14     
+ Partials        446      435      -11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

noleu added 3 commits July 10, 2025 19:10
Addresses an issue where GPU columns were populated with null values.

The changes ensure that default values are used when GPU data is unavailable,
preventing null values from appearing in telemetry tables and tests.
Adds the power draw of individual GPUs to the exported host statistics.

This allows for more detailed analysis of power consumption at the GPU level.
@DanteNiewenhuis DanteNiewenhuis merged commit b2dc97d into atlarge-research:master Jul 15, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants