Danger Talk Sells AI to Enterprises, Not to Consumers

The hypothesis

The claim under test is that when a frontier AI lab announces its new model may enable bioweapons, autonomous cyberattacks, or uncontrolled behaviour, the announcement functions as advertising. The audience hears "so capable it frightens its own builders," which outperforms any benchmark claim, generates free press, makes restricted access desirable, positions the lab as the competent adult in the room, and normalizes compliance costs that smaller rivals cannot absorb.

The structure of frontier-lab communication does fit that shape. It rarely stops at "our model is dangerous." The sequence runs: unprecedented capability, therefore unprecedented risk, therefore uniquely sophisticated measurement, therefore safeguards that make responsible deployment possible. The engine and the seatbelt get sold in the same paragraph.

The opening question

Does describing your own product as dangerous make people more likely to buy it?

That question is answerable, and the answer depends almost entirely on who is doing the buying.

The bottom line

The thesis decomposes into four claims that the evidence supports very differently.

Consumer evidence points against fear selling AI

Making AI salient to a consumer reliably costs you money, and the two best causal estimates are both large.

A field experiment published in Marketing Science put more than 6,200 customers through sales calls. Undisclosed chatbots performed on par with proficient human agents. Disclosing the chatbot's identity before the conversation cut purchase rates by 79.7%, from 0.237 to 0.048, mediated by customers rating the disclosed system as less knowledgeable and less empathetic than the identical system unlabelled (Luo et al., 2019).

A 2026 Journal of Consumer Research paper reaches the same direction at a fraction of the magnitude, which matters for honest reporting. Across 1,135,817 TikTok posts from 8,650 creators plus eight preregistered experiments (N=3,396), AI-generated-content disclosures produced roughly 7% to 8% fewer likes and about 7% less total engagement, conditional on views. The mechanism was not content quality and not generic AI aversion: disclosure signalled lower creator effort, which weakened parasocial connection with the creator (Carney, Riveros and Tully, 2026).

Population attitudes sit in the same place. Half of US adults say increased AI in daily life makes them more concerned than excited, against 10% who are more excited, up from 37% concerned in 2021 (Pew Research Center). By 2026, 67% report little or no confidence in the US government to regulate AI effectively.

Neither experiment tests the specific message "this AI is so powerful it may be dangerous," and that gap is the honest limit of this section. What they establish is a strong prior: AI-ness, made salient, does not behave like prestige advertising in consumer markets.

The forbidden-fruit mechanism is real, and untested on AI

The strongest support for the original intuition comes from psychology rather than from anything about AI.

Bushman and Stack ran three experiments on warning labels for violent television. Warnings increased interest in the labelled programs, the effect was stronger when the label source was authoritative, high-reactance participants were especially drawn to warned content, and warning labels outperformed neutral information labels carrying the same facts (Bushman and Stack, 1996). The causal chain is authoritative warning, then perceived restriction, then reactance, then attraction.

That chain maps cleanly onto "too dangerous to release," "restricted access," "frontier capability," and "available to trusted users only."

A second literature complicates the picture in a useful way. The comprehensive meta-analysis of fear appeals finds they generally do move attitudes, intentions, and behaviour in the intended direction, with very few conditions under which they backfire (Tannenbaum et al., 2015). Fear appeals in that literature are designed to push audiences away from a behaviour, so the finding cuts against a simple "danger attracts" story and in favour of "framing effects are real, direction depends on design."

Nobody has run the AI version. The missing study would expose participants to an identical system under different descriptions and measure perceived capability, curiosity, trust, and willingness to pay separately. Until someone does, "danger framing raises perceived capability" remains a hypothesis with a good pedigree and no direct test.

The enterprise story is where the evidence gets strong

For enterprise buyers, safety talk is procurement assurance rather than fear advertising, and the blockers it addresses are documented.

The OECD/BCG/INSEAD survey of 840 AI-adopting enterprises across the G7 found data privacy, protection and security concerns had limited AI use for 55% of manufacturing and 57% of ICT respondents, with around 40% reporting uncertainty about legal liability for AI-caused damages and a shortage of cloud options guaranteeing regulatory compliance (OECD). UK government adoption research found roughly one in six businesses using AI, with ethical concerns, cost, regulatory uncertainty and data security among the named barriers (GOV.UK).

That produces a clean commercial loop: risk becomes salient, corporate buyers demand controls, and vendors who can evidence controls become easier to procure. ISO/IEC 42001, published in December 2023 as the first certifiable AI management system standard, now shows up in supplier due-diligence packs precisely because it converts a governance claim into an audited one.

The market data is consistent with a safety-forward brand winning the enterprise, without proving it. Menlo Ventures estimates Anthropic took 40% of enterprise LLM API spend by the end of 2025, against OpenAI at 27% and Google at 21%, with the three together at 88% of usage (Menlo Ventures). Anthropic's ascent is attributed in that same analysis mainly to sustained coding performance, so treat this as compatible evidence rather than as a demonstration that safety positioning caused the share shift.

The profitable message here is not "AI is frightening." It is "AI is consequential, and we are competent enough to manage it."

The regulatory moat is visible in the drafting

Frontier-tier rules are written with thresholds that only the largest developers cross, which is the moat mechanism in its most literal form.

The EU AI Act presumes a general-purpose model carries systemic risk once cumulative training compute exceeds 10^25 FLOP, and the Commission's own guidance notes that training at that scale currently costs tens of millions of euros. Providers must notify the AI Office within two weeks of crossing it, then carry systemic-risk assessment, adversarial testing, incident reporting and cybersecurity obligations that ordinary providers do not (European Commission).

California's Transparency in Frontier Artificial Intelligence Act, signed 29 September 2025 as part of SB 53, is more explicit still. It defines frontier models by a 10^26 operation threshold and reserves its heaviest obligations for "large frontier developers," defined as those with annual gross revenue above $500 million (Baker Botts summary).

Layer that onto a market the UK Competition and Markets Authority already flagged as concentrated. Its foundation-model work identified risks from incumbent control of compute, data and talent, from existing routes to market, and from a web of more than 90 partnerships and strategic investments (CMA). Compliance is largely a fixed cost: evaluations, red teams, security programs, audit infrastructure, legal and policy staff. A company spending billions on compute absorbs millions in compliance more easily than a new entrant can, which is the argument developed at length in the AI & Society paper on AI safety and regulatory capture (Springer, 2025).

Two things can be true at once here. The risks can be genuine, and the regulatory response to them can still disproportionately advantage incumbents.

Intent is the part the evidence does not reach

Moving from incentives to motives is where this thesis breaks, and there is direct counter-evidence.

The stated policy positions of the labs have generally argued for threshold-gated oversight rather than blanket regulation. OpenAI's 2023 governance proposal explicitly advocated leaving developers and open-source projects below a significant capability threshold free of licensing and audit burdens (OpenAI). Anthropic's Responsible Scaling Policy ties escalating safeguards to escalating measured capability rather than to AI development generally (Anthropic).

The labs also incur real costs for these positions. Anthropic launched Claude Opus 4 under its ASL-3 Deployment and Security Standard as a precautionary measure, the first model it shipped under that tier (Anthropic). OpenAI treated ChatGPT Agent as High capability in the biological and chemical domain and activated the associated safeguards (OpenAI Preparedness Framework). Most pointedly, the 2026 dispute between Anthropic and the US Department of Defense over whether Claude could be restricted from mass surveillance and autonomous weapons ended with the termination of a contract worth up to $200 million (NPR). A safety commitment that costs a company nine figures of federal revenue is difficult to read as a pure marketing device.

A subtler version of the capture concern survives all of that: whoever defines "frontier," selects the qualifying benchmarks, and decides which practices become mandatory holds real market power. That deserves scrutiny. It is not evidence of manufactured fear.

The asymmetry actually worth watching

Labs have a marketing incentive to emphasize risks that imply intelligence and to underplay risks that imply mediocrity, and this is the part of the original hypothesis that holds up best.

Risks that flatter the technology: autonomous hacking, self-improvement, sophisticated persuasion, bioweapon uplift, models evading control. Every one of them communicates capability.

Risks that embarrass it: hallucinated citations, silently wrong spreadsheet arithmetic, poor long-horizon reliability, prompt injection, data leakage, brittle reasoning, expensive inference, heavy human supervision requirements. Prompt injection is not a speculative concern; it sits at LLM01, the top slot in the OWASP Top 10 for LLM Applications (OWASP), and it is a defect class rather than a superpower.

"Our AI may become intelligent enough to escape human control" reads as a breakthrough. "Our AI invents citations and sometimes botches an invoice" reads as a product complaint. Both appear in system cards. Only one reliably makes the press cycle.

The press cycle is itself part of the mechanism. Mapping of UK AI coverage found nearly 60% of articles indexed to industry products, initiatives or announcements, with 33% of identified sources coming from industry, almost twice the share from academia; the broader review concludes coverage tends to be industry-led and often takes capability claims at face value (Reuters Institute). Lee Vinsel's term for the adjacent failure mode is "criti-hype": criticism that feeds on and inflates the hype it claims to puncture. A warning amplified by a critic still spends the capability claim.

There is a financial-markets analogue with a name and an enforcement record. "AI washing," the exaggeration of AI investment or capability, drew SEC charges against investment advisers in 2024 and now has its own empirical literature measuring how markets price the gap between claimed and actual AI engagement.

What survives: capability-and-control signalling

The model that fits the evidence is not fear marketing. It is a two-part signal carried by a single disclosure.

The capability signal says the technology is consequential enough that governments and scientists treat its consequences seriously. That plausibly generates prestige, coverage, investor interest, talent, and some forbidden-fruit curiosity. Direct causal evidence for the commercial half is thin.

The control signal says the vendor has the institutional capacity to deploy that power responsibly. It answers documented enterprise blockers around security, liability and compliance, and the evidence there is substantially stronger.

Once sophisticated controls become expected or legally required, incumbents with existing capital, compute, legal and policy operations gain a further structural advantage. The mechanism is credible. Deliberate engineering of it is unproven.

The experiment that would settle the interesting part

Randomly assign participants to an identical AI system under four descriptions:

1. Neutral. "Advanced AI assistant."
2. Capability. "One of the world's most capable AI systems."
3. Danger. "Extremely capable; experts believe misuse could cause serious harm, so access is tightly controlled."
4. Assurance. "Independently evaluated, with extensive safety, privacy and security controls."

Measure perceived capability, prestige, curiosity, trust, fear, desire for access, willingness to pay, actual trial behaviour, procurement preference, and support for regulation. Run consumer and business decision-maker samples separately, and separate the restricted-access wording from the harm wording so reactance and risk perception do not travel together.

Predicted from the literature above: danger framing raises perceived capability and curiosity while depressing trust and adoption; assurance framing beats danger framing among enterprise buyers; and the restricted-access phrasing produces a forbidden-fruit effect on its own, independent of the harm claim.

What this means if you are buying AI tools

Read a vendor's risk disclosure as two separate claims and price them separately.

• A capability warning is not a capability benchmark. "Our model crossed a high-risk threshold" is a statement about a lab's internal evaluation policy, not an independent measurement of usefulness for your work. Ask what it scores on the task you actually have.
• Ask about the boring failure modes. Hallucination rates on your document types, prompt-injection handling in any tool with retrieval or browsing, data retention and training terms, and what happens when the model is confidently wrong inside a workflow nobody is checking. These decide whether a deployment works.
• Governance evidence beats governance language. ISO/IEC 42001 certification, published evaluation results, incident-reporting commitments and contractual data terms are checkable. "We take safety seriously" is not.
• Discount both directions of drama. Existential framing and dismissive framing are both cheap to produce. The Reuters Institute finding that coverage is industry-led applies to the vendor deck in front of you as much as to the news article about it.

For the marketing-side version of how AI reshapes where buying decisions actually get made, see our companion report on AI re-routing rather than replacing the web.

Verdict

The defensible version of the thesis is narrow and still interesting. Frontier-risk discourse produces valuable side effects: it signals capability, it turns governance into an enterprise differentiator, and it helps normalize compliance regimes whose fixed costs are easiest for incumbents to absorb. Current evidence does not establish that those effects are the purpose of the rhetoric, and at least one lab has now paid nine figures to hold a safety line.

Sources and further reading