Skip to content

Research note · method

How we report progress: five evidence stages

Every Aporen project is labelled with one of five evidence stages, from hypothesis to released technology. This note defines them and explains why we use them.

Research organizations, especially small ones, are under constant pressure to describe work as further along than it is. A plan becomes a “platform”, a prototype becomes a “breakthrough”, a single favourable run becomes a “result”. We would rather be precise and occasionally unimpressive than impressive and wrong.

Every project on this site is therefore placed on one of five stages.

The stages

Research hypothesis. A stated, falsifiable idea. Nothing has been measured. Writing the hypothesis down is still useful, because it fixes what would count as evidence against it.

Active experiment. Work is running to test the idea. Partial observations exist, but they are incomplete and unpublished, and they should not be quoted as findings.

Prototype. Working software exists and is tested internally. A prototype shows that something can be built; it does not show that the idea behind it holds. A prototype with a full test suite can still be a prototype if its central claim, for example performance on specific hardware, has not been measured.

Validated result. A result has been reproduced under a protocol written down before the measurement, and the method and data are available for review. Differences smaller than run-to-run variation are not results.

Released technology. Validated work that is publicly available and maintained for use by others.

Rules we apply

  1. A project moves up a stage only when the evidence for that stage is public or reviewable.
  2. Missing measurements are reported as missing. A tool that could not take a measurement says so, rather than estimating one.
  3. Each project page separates what is verified from what is not yet done, and records the date its status was last checked against its sources.
  4. Negative results are results. If a hypothesis fails, we will say so in a note like this one.

These stages are deliberately conservative. Most of our current work sits in the first three.