How AI is Increasing Cyber Risk

In July 2025, Sharon Brightwell of Dover, Florida, received a call that no parent wants to hear. Her daughter’s voice came through the phone, crying and distressed. She’d been in a car accident. She’d lost her unborn child. She was facing criminal charges and needed money immediately for legal representation to avoid jail.

Brightwell, overwhelmed by the urgency and her daughter’s obvious distress, acted quickly. She wired $15,000 in cash to a courier the callers sent to her home. Only after the money was gone did she speak to her actual daughter and realize the devastating truth.

VISIT MY SUBSTACK HOMEPAGE FOR MORE FREE ARTICLES >

That hadn’t been her daughter’s voice on the phone. It was an AI-generated clone, artificially created using voice cloning technology. Scammers had likely scraped audio samples from her daughter’s social media posts and used AI to generate a voice that could say anything they wanted. The emotional realism was perfect. The accent matched. The speech patterns were right. Even the distress sounded authentic.

Brightwell’s $15,000 loss represents just one incident in what’s become a massive surge in AI-enabled fraud. Voice cloning attacks alone surged 442 percent between the first and second half of 2024, according to CrowdStrike research. The FBI’s Internet Crime Complaint Center received over 845,000 imposter scam reports in 2024. Many of these now involve AI voice cloning or deepfake technology.

The technology barrier that once protected people has collapsed. Creating a convincing voice clone now requires just three to five seconds of audio and free or low-cost software. No technical expertise needed. No Hollywood-level production budget required. Just a smartphone and publicly available tools.

The Financial Stakes Are Enormous

Deloitte’s Center for Financial Services released research in 2024 predicting that generative AI could enable fraud losses to reach $40 billion in the United States by 2027. That’s up from $12.3 billion in 2023. The compound annual growth rate is 32 percent.

These aren’t speculative projections. They’re based on documented trends in AI-enabled fraud that financial institutions are already experiencing. Deloitte’s assessment includes conservative, base, and aggressive scenarios. Even the conservative estimate puts losses at $22 billion by 2027.

The research highlights three factors driving this surge. First, AI tools that create convincing deepfakes are now cheap and accessible. There’s already an entire cottage industry on the dark web selling scamming software for prices ranging from $20 to thousands of dollars. Second, these tools are getting better at evading traditional fraud detection systems. Third, bad actors can now execute complex fraud schemes at scale using the same or fewer resources than traditional fraud required.

Business email compromises alone could generate $11.5 billion in losses by 2027 in what Deloitte calls an “aggressive” adoption scenario. The FBI counted 21,832 instances of business email fraud in 2022 with losses of approximately $2.7 billion. AI is making these attacks more convincing and harder to detect.

Why Traditional Defenses Aren’t Working

The problem goes beyond deepfakes. AI is transforming multiple categories of cyber risk.

Attackers are using AI to automate significant parts of what security professionals call the “cyber kill chain.” This automation lets them widen the reach and speed of attacks that once required manual effort. A single bad actor with AI tools can now execute attacks that would have required a team.

AI-assisted malware can adapt its behavior to evade detection. Unlike traditional malware that follows predictable patterns, AI-powered threats learn from defensive responses and modify their approach in real time. This makes them harder to stop using conventional security tools.

Social engineering attacks have become more sophisticated. Large language models can craft phishing emails tailored to specific victims with a level of personalization that human attackers couldn’t match at scale. The messages reference actual projects, use appropriate corporate terminology, and mirror the communication style of the person being impersonated.

Voice cloning has become particularly dangerous because it exploits fundamental human psychology. When Sharon Brightwell heard what sounded exactly like her daughter in distress, her rational defenses shut down. The emotional realism of a cloned voice removes the mental barrier to skepticism. That’s why these attacks work even on people who’ve been warned about them.

Then there’s the issue of Shadow AI. Employees using unapproved or unmonitored AI tools can introduce vulnerabilities or leak sensitive data without realizing it. A U.S. Treasury report found that “existing risk management frameworks may not be adequate to cover emerging AI technologies.”

The Attack Surface Is Expanding

AI isn’t just empowering attackers. It’s creating entirely new categories of vulnerabilities.

Prompt injection attacks let adversaries manipulate AI models by feeding them malicious instructions disguised as normal queries. These attacks can cause AI systems to behave in ways their developers never intended.

Developers using AI coding assistants may unintentionally introduce bugs or security flaws. The AI suggests code that looks functional but contains subtle vulnerabilities that traditional code review might miss. These flaws then get deployed into production systems.

Model attacks represent another frontier. Attackers can “poison” AI training data or trick models into misclassifying threats. If a security system relies on AI to identify malicious activity, and that AI has been compromised, the entire defense infrastructure becomes unreliable.

The grandparent scam that targeted Sharon Brightwell represents a particularly insidious evolution. Criminals scrape voice samples from social media platforms like Instagram, TikTok, or YouTube. They clone a young person’s voice. They stage fake emergencies—kidnappings, accidents, legal trouble—to extract money from panicked family members. The elderly are especially vulnerable because they grew up in an era when hearing a familiar voice meant the call was legitimate.

Enterprise Concern Is Surging

A majority of large companies now list AI as a material cyber risk in their financial disclosures. This represents a rapid shift in how corporate leaders view the threat landscape.

Deloitte’s 2024 poll found that 25.9 percent of executives reported that their organizations had experienced one or more deepfake incidents targeting financial and accounting data in the preceding 12 months. More concerning, 50 percent of respondents expected a rise in attacks over the following 12 months.

These aren’t abstract concerns. In May 2025, the FBI issued a warning that “malicious actors” were impersonating senior U.S. officials using AI-generated voice messages targeting current and former government officials and their contacts. The attackers used techniques known as smishing (SMS phishing) and vishing (voice phishing) to establish rapport before attempting to gain access to personal accounts.

Financial institutions report that over 10 percent of banks have suffered deepfake vishing losses above $1 million, with an average loss of $600,000 per incident, according to industry surveys. These attacks often target finance teams in medium-to-large enterprises, where scammers use voice clones of CEOs or CFOs to pressure staff into urgently processing wire transfers or releasing sensitive financial data.

Experts warn that transformative AI capabilities may outpace current safety governance. Organizations are adopting AI faster than they can develop adequate controls. Security teams are trying to defend against threats that didn’t exist a year ago using frameworks designed for a different era.

What This Means

The convergence of sophisticated AI tools, low barriers to entry, and expanding attack surfaces creates a perfect storm for cyber risk. Organizations can’t rely on traditional security measures alone. They need to rethink their entire approach to fraud prevention, employee training, and verification protocols.

Sharon Brightwell did what most parents would do when confronted with a call from their child in distress. She responded with urgency and compassion. Those human instincts, which serve us well in genuine emergencies, now make us vulnerable to attacks that weaponize our emotions against us.

That’s the new reality we’re facing. AI hasn’t just increased cyber risk. It’s fundamentally changed what cyber risk means, exploiting the gap between our evolved trust in familiar voices and faces, and the technology that can now fake them convincingly.