ROCmHub
Reproducible preparation, validation, and benchmarking of open AI models for AMD ROCm.
- Evidence stage
- Evidence stage: Prototype
- Status
- v0.1.0 MVP · Hardware validation pending
- Research domain
- Computing Infrastructure
01 · Problem statement
Running open models on AMD GPUs through ROCm is possible, but preparing a model for that stack is often an ad-hoc process: version mismatches, environment problems, and unverifiable results. Performance numbers are frequently reported without provenance or variance, which makes them hard to compare or reproduce.
02 · Research motivation
Accelerator choice should not be limited by tooling friction. ROCmHub is an attempt to make the path from a model identifier to a verified run on ROCm deterministic and inspectable, and to separate clearly what was executed, what was measured, and what was only configured.
03 · Technical direction
- A
Deterministic preparation
Model metadata resolved to immutable commit hashes, deterministic build plans, and standalone launch scripts.
- B
Explicit evidence states
Runs are labelled with states such as CONFIG_ONLY or NOT_MEASURED when a step could not be executed, instead of producing estimated numbers.
- C
Validation before benchmarking
Output corruption checks and token-agreement scoring gate any performance measurement.
- D
Statistical benchmarking
Latency and throughput percentiles with dispersion checks based on median absolute deviation and environment fingerprinting.
04 · Current verified status
- 01 Hypothesis
- 02 Experiment
- 03 Prototype (current stage)
- 04 Validated
- 05 Released
Prototype: Working software exists and is tested internally. Its central claims are not yet validated.
Verified
- Version 0.1.0 (MVP) is published as open source under the Apache 2.0 licence.
- Implemented: CLI, environment diagnostics, model metadata inspection, deterministic build planning, validation engine, benchmark harness with dispersion checks, checksummed artifact bundles, a local API, and a web interface.
- Tested in continuous integration on Linux (Python 3.9–3.12, without GPUs) and in diagnostic and dry-run modes on macOS.
Not yet done or demonstrated
- Execution on physical AMD hardware has not yet been performed. No AMD performance results exist.
- The first hardware target (Radeon RX 7900 XTX, gfx1100) is documented but not yet run.
Status source: Public repository README and release notes (github.com/RocmHub/RocmHub). Checked .
05 · Known challenges
-
Hardware access
Validation on physical AMD GPUs is the main open dependency.
-
Stack drift
ROCm, PyTorch, and driver versions change frequently, and reproducibility has to survive that.
-
Coverage
The initial runner targets one inference path (PyTorch with HIP) and causal language models.
06 · Next milestones
- M1 First physical run on an AMD Radeon RX 7900 XTX with a small open model, following the published first-run procedure.
- M2 A public, checksummed artifact bundle from that run, including the variance data.
- M3 Extension of the validation suite to additional model architectures.
Milestones are planned work, listed in intended order. They are not dated commitments.
07 · Public resources
References
External work this project builds on or compares against. Listing does not imply endorsement or affiliation.