Skip to content
ROCMHUB Open-source tooling · v0.1.0

ROCmHub

Reproducible preparation, validation, and benchmarking of open AI models for AMD ROCm.

Evidence stage
Evidence stage: Prototype
Status
v0.1.0 MVP · Hardware validation pending
Research domain
Computing Infrastructure

01 · Problem statement

Running open models on AMD GPUs through ROCm is possible, but preparing a model for that stack is often an ad-hoc process: version mismatches, environment problems, and unverifiable results. Performance numbers are frequently reported without provenance or variance, which makes them hard to compare or reproduce.

02 · Research motivation

Accelerator choice should not be limited by tooling friction. ROCmHub is an attempt to make the path from a model identifier to a verified run on ROCm deterministic and inspectable, and to separate clearly what was executed, what was measured, and what was only configured.

03 · Technical direction

  1. A

    Deterministic preparation

    Model metadata resolved to immutable commit hashes, deterministic build plans, and standalone launch scripts.

  2. B

    Explicit evidence states

    Runs are labelled with states such as CONFIG_ONLY or NOT_MEASURED when a step could not be executed, instead of producing estimated numbers.

  3. C

    Validation before benchmarking

    Output corruption checks and token-agreement scoring gate any performance measurement.

  4. D

    Statistical benchmarking

    Latency and throughput percentiles with dispersion checks based on median absolute deviation and environment fingerprinting.

04 · Current verified status

  1. 01 Hypothesis
  2. 02 Experiment
  3. 03 Prototype (current stage)
  4. 04 Validated
  5. 05 Released

Prototype: Working software exists and is tested internally. Its central claims are not yet validated.

Verified

  • Version 0.1.0 (MVP) is published as open source under the Apache 2.0 licence.
  • Implemented: CLI, environment diagnostics, model metadata inspection, deterministic build planning, validation engine, benchmark harness with dispersion checks, checksummed artifact bundles, a local API, and a web interface.
  • Tested in continuous integration on Linux (Python 3.9–3.12, without GPUs) and in diagnostic and dry-run modes on macOS.

Not yet done or demonstrated

  • Execution on physical AMD hardware has not yet been performed. No AMD performance results exist.
  • The first hardware target (Radeon RX 7900 XTX, gfx1100) is documented but not yet run.

Status source: Public repository README and release notes (github.com/RocmHub/RocmHub). Checked .

05 · Known challenges

  • Hardware access

    Validation on physical AMD GPUs is the main open dependency.

  • Stack drift

    ROCm, PyTorch, and driver versions change frequently, and reproducibility has to survive that.

  • Coverage

    The initial runner targets one inference path (PyTorch with HIP) and causal language models.

06 · Next milestones

  1. M1 First physical run on an AMD Radeon RX 7900 XTX with a small open model, following the published first-run procedure.
  2. M2 A public, checksummed artifact bundle from that run, including the variance data.
  3. M3 Extension of the validation suite to additional model architectures.

Milestones are planned work, listed in intended order. They are not dated commitments.

07 · Public resources

References

External work this project builds on or compares against. Listing does not imply endorsement or affiliation.