Skip to content
ZERO Research initiative

ZERO Research Lab

Investigating how much of language-model inference is necessary computation.

Evidence stage
Evidence stage: Research hypothesis
Status
Early research · Planning
Research domain
Intelligence Systems

01 · Problem statement

Running a language model costs roughly the same amount of computation for every token, regardless of how difficult that token is to predict. As models are deployed more widely, inference, rather than training, becomes the dominant and recurring cost. Many existing efficiency techniques require retraining, architecture changes, or model-specific kernels, which limits who can use them and on which models.

02 · Research motivation

There is a substantial research literature showing that parts of transformer computation can be skipped, approximated, or reused for some inputs: early exits, layer skipping, speculative decoding, and cache management among them. Most of these results are tied to a specific model family or training procedure. ZERO asks whether the underlying redundancy can be detected and exploited in a way that is independent of the model, so that existing models benefit without retraining.

Working hypothesis · untested
For a meaningful share of inputs, a measurable fraction of inference computation in existing language models is redundant and can be identified at run time cheaply enough that skipping it reduces total cost, without retraining and without an unacceptable loss in output quality. This is an untested hypothesis; it may be false, or true only under narrow conditions.

03 · Technical direction

  1. A

    Map the redundancy

    Survey and classify where redundant computation has been reported (layers, attention heads, tokens, repeated prefixes) and under which conditions.

  2. B

    Define the measurement before the method

    Fix quality metrics, baselines, hardware, and the accounting of detection overhead before any optimization is attempted, so results cannot be selected after the fact.

  3. C

    Reproduce, then extend

    Reproduce established techniques on small open models to calibrate the harness, then test model-agnostic variants against them.

  4. D

    Report negative results

    Publish what does not work with the same care as what does. A clear negative result is a valid outcome for this programme.

04 · Current verified status

  1. 01 Hypothesis (current stage)
  2. 02 Experiment
  3. 03 Prototype
  4. 04 Validated
  5. 05 Released

Research hypothesis: A stated, falsifiable idea. No experimental evidence yet.

Verified

  • The research direction and scope are defined.
  • The initiative is in planning. Literature review is the first work item.

Not yet done or demonstrated

  • No experiments have been run.
  • No computational savings have been demonstrated.
  • No code, datasets, or models have been released.

Status source: Project brief; no code, experiments, or data exist yet. Checked .

05 · Known challenges

  • Detection overhead

    Deciding what to skip costs computation; the decision must be cheaper than the work it avoids.

  • Quality measurement

    Small degradations in output quality are hard to measure and easy to overlook with weak evaluations.

  • Model-agnosticism

    Techniques that generalise across architectures may capture less of the available savings than model-specific ones.

  • Hardware dependence

    Skipped operations do not always translate into wall-clock savings on real accelerators, because of batching and memory-bandwidth effects.

06 · Next milestones

  1. M1 Literature review and a written taxonomy of reported redundancy in transformer inference.
  2. M2 A documented measurement protocol covering quality metrics, baselines, hardware, and overhead accounting.
  3. M3 Reproduction of at least one established technique on a small open model to validate the harness.
  4. M4 A first falsifiable experiment, published as a research note regardless of outcome.

Milestones are planned work, listed in intended order. They are not dated commitments.

07 · Public resources

No public resources yet. This page will link to them when they exist.