[ Evidence-led insight ]

How to tell whether an AI assistant's cited sources and references are real before I rely on or republish them

A guide on mapping AI citation failure modes to MPS/0.1 claim tags and using the free MPS Auditor to generate claim-by-claim provenance records before publication.

By Mayone Maha Rajan · Maha Strategies LLC ·

Direct answer

The reader leaves with a concrete, repeatable way to separate real, checkable citations from plausible-but-fabricated ones, and knows which claims to hold as unverified until a human confirms the primary source.

Maha method

Maha maps common AI-citation failure modes (fabricated references, real-but-wrong DOIs, uncited statistics) to the MPS/0.1 claim tags and pairs each with a reviewer action, then runs a real AI passage through the free MPS Auditor to produce a claim-by-claim provenance record. The Auditor labels each claim's evidentiary status for a human to check; it does not determine truth or certify accuracy.

[ Evidence artifact ]

Free MPS Auditor Tool

This artifact supports an editorial workflow; it does not independently establish factual truth.

Limits

The MPS Auditor labels a claim's evidentiary status for human review; it does not determine factual truth, certify overall accuracy, or eliminate the need for manual verification of primary sources.

Sources

  1. Fabrication and errors in the bibliographic citations generated by ChatGPT

    Peer-reviewed Scientific Reports study finding 55% of GPT-3.5 and 18% of GPT-4 generated citations were fabricated, with many real citations still containing substantive errors; supports the claim that AI-produced references cannot be trusted without checking the primary source.

  2. Hallucinating Law: Legal Mistakes with Large Language Models are Pervasive

    Stanford RegLab research summary reporting legal hallucination rates of 69%–88% on specific queries to leading models; supports that AI confidently produces unverifiable or wrong citations, a high-stakes case for pre-publication verification.

  3. AI Search Has a Citation Problem

    Tow Center study of eight AI search tools finding over 60% of responses cited sources incorrectly and often misattributed content; supports that even 'grounded' AI search mislabels its sources, so citations must be independently checked.

Need a source-tagged analysis for a decision? Explore consulting.