The Next National Security Challenge Is Research Integrity

A European cybersecurity agency. A national AI strategy. A flagship White House health report. Between May 2025 and April 2026, all three released official documents built partly on research that did not exist.

The European cybersecurity agency ENISA published threat landscape reports last year that outside researchers found included AI-generated or otherwise erroneous references. By one count, one report contained 26 incorrect footnotes out of 492; the agency later issued a revised version that corrected some links. In South Africa, the government withdrew its draft national AI policy after media reporting found that at least six of its sixty-seven academic citations were fabricated. In the United States, a government Make Our Children Healthy Again assessment cited studies that did not exist and misattributed others. The Washington Post separately found that dozens of the report’s footnotes carried an “oaicite” marker, a fingerprint of OpenAI-generated citations.

These episodes occurred under different governments, in different regions, and on very different policy questions. Yet they reveal the same institutional vulnerability: unreliable AI-assisted research that can undermine national policy with bad information. As governments around the world are racing to adopt artificial intelligence, most attention has gone to securing the models themselves: reducing safety risks and hardening cyber defenses against attack. Far less has gone to a quieter shift—the way AI is changing whether what is presented as knowledge itself can be trusted.

The encouraging news is that AI detection systems are maturing. AI text detectors such as Pangram, OriginalityAI, GPTZero and RoBERTa perform well on some metrics, though these tools remain imperfect. The more pressing question is what current methods are missing, how these failures pose risks, and how institutions can adapt to treat the integrity of information as a security concern in its own right.

For decades, governments have built institutions to produce trustworthy knowledge: professional civil services, federally funded research and development centers, intelligence tradecraft, and layered analytical review. Those systems assumed that humans generate institutionally verified knowledge, but generative AI is unsettling that assumption, offering great promise for governments to access information to produce analysis at remarkable speed while introducing new challenges for information credibility and verification.

The fabrication of knowledge is an increasing concern across research fields. In a 2023 study in Scientific Reports, researchers who asked ChatGPT to generate literature reviews found that 55 percent of GPT-3.5 citations and 18 percent of GPT-4 citations were entirely fabricated, pointing to no real work. Even among the citations that did point to real papers, 43 percent of GPT-3.5’s and 24 percent of GPT-4’s contained substantive errors. The problem persists with newer models. A 2025 experimental study in JMIR Mental Health found that GPT-4o fabricated roughly one in five of the citations it generated, with nearly two-thirds either fabricated or inaccurate once bibliographic errors were counted, and that fabrication was worse for specialized or less prominent topics than for well-known ones. That last finding should worry policymakers most, because the obscure, technical subjects where independent verification is hardest are precisely where fabrication is most likely.

AI's presence in research continues to set new questions for scientific inquiry. A 2025 analysis in Science Advances that tracked sudden shifts in word usage across more than 15 million biomedical abstracts estimated that at least 13.5 percent of 2024 abstracts had been processed with a language model, rising toward 40 percent in some fields, and Stanford researchers found a comparable footprint inside peer review itself. These are lower-bound estimates, and they say nothing on their own about accuracy. What they establish is scale: AI-assisted text has entered the scholarly record in volume, often without disclosure. More than 10,000 papers were retracted in 2023, a record driven largely by industrialized paper mills that sell fraudulent manuscripts; roughly 8,000 came from a single publisher, suggesting AI is expanding the ability to produce and distribute bad research.

When research is corrupted, so is the security of the nations whose policies depend on it. Fabricated citations and confidently incorrect synthesis can misinform policy responses, skewing battlefield threat assessments and the related policymaking that keeps societies secure. The risk is that no one catches a fabrication in time in policy areas where decisions stick. US courts have already seen evidence of orders resting on evidence from cases that did not exist, driving a binding ruling before the appeals court vacated it. A courtroom has an appeals process that can undo the damage, but a fabricated line inside a threat assessment, acted on in real time, has no such backstop.

Consider arms control and nuclear security, where verification regimes exist precisely because confidence in evidence is itself a strategic asset. While there is not a public case of AI-generated citations undermining a nuclear assessment, the risk remains and the consequences are catastrophic. Concern about AI in nuclear decision-making has already reached the highest levels: in November 2024, the US and China agreed that humans, not AI, should control nuclear weapons, and defense analysts stress that leaders must be able to trace what role AI played in shaping the intelligence and recommendations they act on.

The same tension runs through intelligence and defense, where the direction of travel is faster adoption. In the US, civilian agencies are now directed to govern AI through OMB Memorandum M-25-21, which prioritizes accelerating adoption while keeping minimum safeguards for high-impact systems. Issued under President Donald Trump’s Executive Order 14179, it replaced earlier guidance. Defense systems are governed separately but point the same way. These efforts are sensible. AI can translate documents, surface patterns across vast datasets, and compress months of research into minutes. But speed cannot substitute for verification, and as synthesis accelerates, the institutions that evaluate evidence only become more important. Adoption is already concrete: thousands of analysts across the eighteen US intelligence agencies use Osiris, a CIA-built generative AI tool for open-source analysis, even as the agency’s own chief technology officer has cautioned that today’s models still hallucinate. The risk of moving faster is measurable: in one Stanford-based study, every one of five large language models tested in simulated military and diplomatic crises showed tendencies toward escalation, including in scenarios that began neutrally.

Diplomacy and international development show other dangers. Foreign ministries, aid agencies, and international organizations increasingly depend on rapid synthesis of country reports, humanitarian assessments, and local-language research. AI is genuinely promising to widen access to scholarship, ease language barriers, and help overstretched institutions read complex environments faster. Yet these are also the settings where verification capacity is weakest as resources are dramatically dipping, making fabrication risks higher. A hastily synthesized country assessment can move development finance, humanitarian resources, and diplomatic engagement, and getting it wrong carries security consequences at home when instability spreads from abroad.

Well governed, generative AI may become one of the most valuable research tools for governments in recent history. The task remains to build verification that keeps pace with generation, and the encouraging news is that many of the building blocks already exist across journals, publishers, and civil society.

First, governments can do more to treat citation verification as routine infrastructure rather than a final courtesy check. Systems currently are based on reactive, patchwork policies and rely heavily on voluntary disclosures and goodwill. Every reference in an AI-assisted government report should trace to a credible source, and much of that work can be automated. Publishers already run reference and integrity screening at scale through shared systems like the STM Integrity Hub and commercial paper-mill detectors, and open civil-society efforts such as the Black Spatula Project have begun using AI itself to flag errors, including calculation and factual mistakes, in published work. Governments that commission research could require comparable audits before publication, much as they require financial audits before publishing budgets.

Second, model developers, researchers and analysts involved in policy-relevant research can also work to change how the tools are used, not only how outputs are checked. AI research assistants should be configured to ground their claims in retrieved, linkable sources rather than generate citations from memory, the single design choice most associated with lower fabrication. Where a model cannot produce a verifiable source, that gap should surface to the user rather than be smoothed over with an invented one.

Third, policymakers should set clearer, more consistent standards that reward disclosure of AI use in research. Readers do not need a full prompt history, but they should know enough to calibrate trust, the same logic behind the watermarking and provenance standards spreading elsewhere in the information environment.

Fourth, governments should build capacity for evidentiary verification in the age of AI. Detection tools are improving but remain imperfect, and false positives can unfairly damage reputations. High-consequence documents should develop stronger responsibility and capacity for source-checking and uncertainty assessment. That means investing in the people who can do it: the public-interest technologists and trained reviewers who can sit between policymakers and the models they increasingly rely on.

These options are far from radical, though sorely underdeveloped in policy spaces and still represent a patchwork. Scientific journals and national-security publications already lean on source verification, disclosure, and editorial review to protect trust. Nature requires authors to disclose certain uses of generative AI, and War on the Rocks, through its Cogs of War vertical, rejects submissions it suspects are machine-written.

None of this means governments should categorically slow AI adoption. AI is already helping analysts process multilingual documents, identify emerging risks, and expand access to evidence with potential to improve policy responses. The challenge is ensuring that institutions modernize verification as quickly as they modernize generation.

For decades, policymakers have treated critical infrastructure as physical: power grids, communications networks, transportation systems, and satellites. In the age of generative AI, another kind deserves equal attention: the institutions that determine what governments believe to be true.

The race to deploy AI across government is already underway. Governments have spent years preparing for attacks on digital infrastructure. The next infrastructure they must protect is the integrity of the evidence on which national decisions rest. In the age of AI, research integrity has become critical infrastructure.