How much could content labeling help with AI-enabled advertising scams?

Advertising scams trick people into losing money or personal data, such as by selling products that look nothing like what was advertised, or never shipping them at all. AI systems could potentially facilitate these scams, for example by generating fake product images or celebrity endorsements. Motivated in part by these concerns, some jurisdictions are requiring that AI content be labeled as AI-generated (e.g. the EU). One hope is that these labels will help people spot untrustworthy ads.

But how effective would content labeling be? Unfortunately, we don’t have much good data on this question, or on how much AI is contributing to advertising scams.

The data that does exist suggests content labeling could be cost-effective, but only if AI ends up driving a large share of scam losses. And even if all advertising scams came to involve AI, content labeling would still fail to prevent potentially tens of billions in losses per year.

An alternative worth exploring is to limit the supply of scam ads—for example, through mandatory advertiser verification on digital platforms—in addition to trying to change how consumers respond to them.

There’s no good data on losses from advertising scams. We’ll instead derive a rough upper bound based on related statistics. Since we’re interested in the rough order of magnitude, I’ll ignore inflation to keep things simple.

This paper ran a survey to estimate losses from cybercrime. The category of cybercrime most relevant to advertising scams is non-delivery: when the scammer is paid without delivery of the product or service. Because non-delivery could happen without ads, we’ll treat the following estimate as a rough upper bound. The survey-weighted mean loss was $116 (standard error = $22).1 The survey was performed in 2020, when the US adult population was about 260 million. We thus estimate an upper bound on US losses from advertising scams in 2020 to be:

Even the lower end of this estimate is enormous. As a reference point, in 2025 the FTC had a budget of $425 million.2

As a spot check of the order of magnitude, the FTC recorded $432 million in reported losses from online shopping and negative reviews fraud in 2024. Assuming that about 5% of people report their losses, true losses in this category are about $8.6 billion. The category imperfectly overlaps with non-delivery: the latter would also cover fraudulent services but excludes negative reviews. But the estimate is roughly within the same order of magnitude as $30 billion.

As another spot check, Meta projected roughly $16 billion in 2024 revenue from ads promoting scams and banned goods. We’re not interested in banned goods, but some portion of the $16 billion is a rough lower bound on losses because scams are presumably profitable. Meta also estimated that “the company’s platforms were involved in a third of all successful scams in the U.S.” Multiplying the full $16 billion by 3 gives roughly $48 billion—an overestimate, but in the same ballpark as the $30 billion derived above.

We don’t have great empirical data on how much AI is contributing to advertising scams. There are some evaluations of (mostly phishing) scam capabilities and some reports of AI use in scams, but such data is not enough to estimate monetary damages.

We can also check whether notable AI releases have caused any uptick in scam losses. Based on FTC data on reported scam losses over time, I plot the growth in scam losses across three categories of scams:

• Business Imposters

• Advance-Fee Credit (e.g. tricking somebody to pay upfront for a loan)

• Miscellaneous Investments & Investment Advice

Overall, it’s hard to attribute growth in losses to past AI releases. For example, ChatGPT was released in November of 2022, but growth for business imposter and investment scams was lower in 2023 than 2022, and losses actually decreased for advance-fee credit scams. Similarly, DALL-E 2 and 3 don’t show up (released 2022 and 2023 respectively).

This data is hard to interpret for several reasons. Even if AI was having an impact, it’s possible that other preventative measures (e.g. AI use in law enforcement) could have compensated. It could also be that AI systems were not yet capable enough, or that scammers had not yet found the best ways to use AI. Finally, any increase in growth could be due to factors other than AI, such as proliferation of scam techniques or increased saliency of particular scam categories.

There’s not much more we can do here. We’ll just need more data to understand how much AI could contribute to scam losses.

Even without good data on AI’s contribution to advertising scams, we can still try to understand how effective content labeling could be. Since some jurisdictions are already implementing content labeling requirements, this exercise could help us gauge how much to expect from such efforts and whether to pursue complementary approaches.

Because we’re interested in order-of-magnitude estimates, I’ll keep things simple by using nominal dollars throughout rather than adjusting for inflation.

Content labels tell viewers something about the origins and history of a piece of content. For the purposes of advertising scams, the most important information is:

• Whether the content was generated by an AI system.

• Whether and how the content was modified, such as image cropping.

These labels can be applied at different points in time. AI providers can watermark or tag content at the point of generation. Social media platforms can then detect these tags, or run their own classifiers, and show labels to users. Meta already labels AI-generated images on its platforms (although such labeling does not apply to all ads).

To be conservative, we’ll try to overestimate the costs of implementing content labeling. The costs come from:

• AI companies signing content labels to show they are authentic

• AI companies applying watermarks to AI-generated content

• Social media platforms creating content labels as necessary (e.g. watermark detection, classifiers for content that is not labeled) and displaying labels to users

We assume signing content labels involves implementing C2PA (the de facto standard for AI content), which requires purchasing a signing certificate. This cost is not totally transparent,3 but the cost of analogous certificates is on the order of around $1000/year max. Assuming integration and maintenance require a dedicated 5-person team at $400k/year per engineer, the cost is about $2 million/year per company. If five frontier AI companies have to implement content labeling (OpenAI, Anthropic, Google DeepMind, xAI, Meta), the industry-wide cost of signing would be $10 million/year.

Adding text metadata is basically free once you have a signing certificate.

For watermarking, Google’s SynthID imposes a 0.57% latency increase with a 7B-parameter model. The absolute latency cost does not scale with model size, meaning that the percentage latency increase should decrease with model size. We thus use 0.57% as an overestimate of the latency increase due to watermarking.

To obtain the monetary cost of applying watermarks, we need to know the amount spent on inference. Epoch estimates that OpenAI inference spend in 2024 was $2 billion. Unfortunately, there don’t seem to be good estimates of inference costs at other AI companies. As an alternative, we extrapolate from OpenAI market share. In 2024 OpenAI had about 28%4 of the enterprise market share (from a survey) and 80% consumer market share (estimated by site visits). Dividing $2 billion by each share gives us an estimate of total enterprise inference spend ($7.1 billion) and total consumer inference spend ($2.5 billion). Adding these numbers gives $9.6 billion, an overestimate since OpenAI’s $2 billion covers both markets. We thus get an upper bound on the industry-wide cost of watermarking: $9.6 billion * 0.0057 ~= $55 million.

Putting it all together, we estimate that content labeling costs the AI industry $65 million / year.

In 2021, Meta spent approximately $5 billion on “safety and security”, which includes content moderation and other functions. Assuming that Meta’s platforms “were involved in a third of all successful scams in the U.S.”, we multiply this number by 3 to cover all digital platforms: $15 billion.

The amount spent on content moderation already includes some scam detection, since Meta lists scams among its 20+ existing policy areas. AI content labelling would add additional work but seems marginal to existing moderation infrastructure. Thus, we estimate additional costs from content labeling at about 1/20th of the total: $750 million/year.

The total estimated cost of content labeling is therefore about $800 million/year.

Let’s think of a scam as composed of three components: some people are targeted, some people engage with the scam, and some of those people lose money.

In principle, content labels could reduce losses in two ways:

1. Reducing engagement: For example, someone sees that a product listing is from a suspicious source and decides not to engage.

2. Reducing loss per person engaged: Labels might increase the skepticism of people who engage, causing them to invest less money or pull out sooner.

Content labels do not seem likely to reduce the number of people targeted, since targeting happens before a person ever sees the content.

What’s the effect size of the two remaining mechanisms?

Unfortunately, there is no empirical data on how content labels affect susceptibility to scams. As a small consolation, there is some work on how AI-generated content labels affect self-reported engagement and trust.

For engagement, one study found that click-through intent decreased by about 0.3 points on a 7-point scale when AI labels were present. Another found that engagement decreased by a maximum of 0.2 points on a 5-point scale.

For trust, two studies found that labeling content as AI-generated reduces perceived trustworthiness or accuracy by at most about 0.35 points on a 5-point Likert scale.5 However, at least one study found no significant effect of AI labels on message persuasiveness.

How do we convert changes on a Likert scale to percent reductions in scam susceptibility? In general, we’re out of luck. Likert scales are ordinal, and numerical differences between different positions on the scale are not necessarily meaningful. There could also be threshold effects where small changes in ratings trigger large behavioural shifts. On the other hand, stated beliefs can often differ from actual behaviour.

Keeping these caveats in mind, I’ll attempt some rough estimates anyway with some strong assumptions. I’ll assume that reductions in trust directly translate into reductions in loss per person engaged, and that self-reported belief translates into behaviour.

We use two methods to derive a range of estimates for the percentage reductions.

1. One option is to divide the changes by the baseline Likert ratings. The intuition here is to treat the Likert score as if it was proportional to the quantity in question.

   1. For one of the engagement studies, the baseline scores were around 4.5-5 on a 7-point scale. For the other one, the baseline scores were around 1.8-2.5 on a five-point scale. We thus obtain roughly a 7-11% reduction in engagement.

   2. For trust, the smallest baseline ratings in the studies were around 2-3 on a 5-point scale, so a 0.35-point decrease corresponds to roughly a 12-18% reduction. If we account for the study that found no significant effect of labels on message persuasiveness (0% effect), we obtain a range of 0-18% for reductions in trust.

2. Another option is to divide the changes by the full range of the Likert scales. Intuitively, we’re normalizing by treating the change as a fraction of the scale width. As a result, this method is more conservative than the first one.

   1. Doing the calculation gives us roughly a 4% reduction in engagement and a 0-7% reduction in trust.

We construct a combined range by taking the minimum and maximum reductions across the two methods for both engagement and trust. Doing so, we obtain a 4-11% reduction in engagement and a 0-18% reduction in trust/loss per person engaged. Combining these ranges using our simple model above, the reduction in losses for AI-enabled advertising scams would be in the range 4-27%.

Again, these estimates are extremely rough. The effect size might also be much smaller in practice because:

• Users might not read or act on the labels.

• Scammers can evade labels, such as by using open weight models with no watermarking or producing content without AI.

• If legitimate advertising had content labels but scam advertising did not, scam ads could look more convincing.

Let’s now look at cost-effectiveness. The 4-27% reduction applies only to the share of advertising scam losses involving AI content, but we don’t know what this share is. The table below shows cost-effectiveness ratios (benefits/costs) under different assumptions about this share.

Assuming a 4% reduction in losses, content labeling only starts becoming cost-effective when more than 50% of total losses are due to AI. Assuming a 27% reduction, the same threshold is close to 10%. It’s therefore somewhat plausible that content labeling could be cost-effective for advertising scams, especially since we likely overestimated the cost of implementing it.

There are some caveats to this analysis:

• Content labeling could look worse than estimated because the actual reduction in losses could be substantially smaller than 4-27%. Furthermore, if losses from advertising scams are smaller than $30 billion, the benefits of content labeling shrink while the implementation costs remain roughly the same.

• Content labeling could look better than estimated because it could provide other benefits, such as improving online trust in general.

Let’s also think about the best-case scenario for content labeling. With a maximum reduction of 27% and assuming that all advertising scams are AI-enabled, content labeling would prevent 0.27 * $30 billion ~= $8 billion in losses. This amount is substantial. However, it still leaves $22 billion in losses unaddressed.

What could help us address the remaining tens of billions in losses from advertising scams?

Content labeling targets the demand side of advertising scams by helping users recognize untrustworthy content. Alternatively, we could target the supply side by getting digital platforms to take down scam ads. Digital platforms could have a lot of influence: Meta’s staff estimated that the company’s platforms were involved in “roughly one-third of all successful U.S. scams”.

Unfortunately, there is an incentive problem. Internal Meta documents, reported on by Reuters, show that Meta projected roughly $16 billion in revenue (10% of its total 2024 revenue) from ads promoting scams and banned goods. Catching scammers is an imperfect science, but according to these documents:

the company only bans advertisers if its automated systems predict the marketers are at least 95% certain to be committing fraud [...] If the company is less certain – but still believes the advertiser is a likely scammer – Meta charges higher ad rates as a penalty.

Meta also sets revenue guardrails:

[T]he team responsible for vetting questionable advertisers wasn’t allowed to take actions that could cost Meta more than 0.15% of the company’s total revenue. That works out to about $135 million out of the $90 billion Meta generated in the first half of 2025.

The upshot is that large platforms can have a financial interest in tolerating scams.

What should we do about that? One option is to require more transparency, such as by requiring platforms to report on scam detection thresholds and ban rates. More information of this kind could potentially improve accountability. The Reuters investigation above has already motivated new legislation.

Another option is requiring platforms to verify the identity of advertisers, such as through government-issued ID. There are two potential benefits: it gives law enforcement someone to trace when scam reports come in, and it makes it harder for bad actors to misrepresent themselves (e.g., foreign operations posing as domestic companies). The bipartisan SCAM Act, introduced in Congress in February 2026, would require such verification along with broader obligations for platforms to prevent fraudulent ads.

There’s some evidence that advertiser verification works. In Taiwan, a combination of mandatory verification and external systems to scan and report scam ads “brought down rates of scam ads involving investments by 96% and identity impersonation by 94%”. A caveat is that banned scammers could try other jurisdictions or platforms. The same article notes: “Meta experiments showed that while fraudulent ads decreased in Taiwan after the rule change, its algorithms simply rerouted them to users in other markets.”

Another potential approach to investigate is to make platforms financially liable for scam losses, which would require revisiting Section 230 in the U.S. Platforms will likely be reticent because of their potential costs are large: one internal Meta analysis estimated that “possible regulation in Europe and Britain that would make Meta liable for its users’ scam losses could cost the company as much as $9.3 billion”.

There are a lot of areas of further work.

First, we need better data on the effectiveness of content labeling. Existing studies rely on self-reported intentions, but experiments measuring actual behavior (e.g., real click-through and purchase decisions) would be far more informative.

Second, platform regulation could likely be improved. Options range from transparency requirements and mandatory advertiser verification to liability for scams.

Finally, this post focused on scams, but content labeling could also strengthen online trust, which could improve the cost-effectiveness case for content labeling.

Thanks to the following people for feedback that greatly improved this piece: Kamilė Lukošiūtė, Aidan Homewood, Matthew van der Merwe, Elias Groll, Markus Anderljung, and Saket Kumar. All errors are my own.