frameworkNumerical methods

Convergence, precision, and error

Separate approximation error, floating-point behavior, input uncertainty, and model discrepancy.

Working definition

Convergence describes whether an approximation approaches a limiting value as resolution or iteration changes. Precision describes numerical representation or repeatability, while error is deviation from a reference quantity. Reproducible software must not collapse truncation, roundoff, measurement uncertainty, and model discrepancy into one number.

Notation

eₙ = xₙ − x*|eₙ₊₁| ≤ C|eₙ|ᵖ

Assumptions

  • A target quantity and reference meaning are defined.
  • Stopping criteria are scale-aware.
  • Arithmetic and library versions are recorded.

Invariants

  • More printed digits do not imply lower error.
  • Convergence to a value does not imply convergence to the correct model.
  • Tolerance is not identical to uncertainty.

Reproducible procedure

  • Identify error sources before computation.
  • Run refinement and independent-reference checks.
  • Report precision, tolerance, residual, and uncertainty separately.

Error and boundary controls

  • Catastrophic cancellation can dominate.
  • Ill-conditioned problems amplify tiny perturbations.
  • Unknown model error cannot be inferred from solver residual alone.

What this does not establish

Numerical agreement establishes implementation consistency only within tested conditions; it cannot validate a symbolic or causal claim.

Explicit applications

1 cross-domain bridges

Celestial factscalculation

Calculation conformance and precision

Separate solver tolerance, floating-point precision, source uncertainty, and convention disagreement.

Inputs

  • implementation output
  • independent reference
  • declared tolerances

Outputs

  • residuals
  • pass or explain verdict
  • disagreement category

Transformation: Compare continuous values before classifications and attribute discrepancies.

Limit: Conformance within tolerance validates implementation behavior, not interpretive claims.

Open connected system →

Authoritative references

  1. [1]DLMF Chapter 3: Numerical Methods · National Institute of Standards and Technology

    Establishes: Reference definitions, algorithms, convergence conditions, and error terms for interpolation, quadrature, differentiation, and nonlinear equation solving.

    Boundary: A numerical method is reliable only under its stated regularity, conditioning, precision, and convergence assumptions; the reference does not validate any domain interpretation.

  2. [2]NIST Technical Note 1297: Guidelines for Evaluating and Expressing Measurement Uncertainty · National Institute of Standards and Technology

    Establishes: A measurement framework for identifying uncertainty components, combining standard uncertainties, and reporting expanded uncertainty with declared coverage.

    Boundary: Reported uncertainty describes the measurement model and included components. It is not a guarantee that all systematic errors or model inadequacies were found.

Related mathematical concepts