Prospective evaluation and falsifiability protocol

Multiplicity and stopping-rule protocol

Prevent many attempted claims or repeated peeking from manufacturing a positive result.

Required inputs

Hypothesis family

Primary comparison

Adjustment method

Fixed or sequential stopping rule

Maximum sample

Ordered execution

Work the protocol

  1. 1

    Enumerate every planned test.

  2. 2

    Designate the primary test.

  3. 3

    Freeze multiplicity adjustment.

  4. 4

    Freeze when data collection stops.

  5. 5

    Report all tests and looks.

Outputs

  • Hypothesis-family manifest
  • Adjusted results
  • Stopping audit

Done only when

  • Every planned test is accounted for.
  • Adjustment is reproducible.
  • Stopping follows the locked rule.

Fail closed

Refusal conditions

  • Only the best frame is reported.
  • Data collection stops after a favorable look.
  • Exploratory tests are called confirmatory.