More than 40 countries are holding national elections in 2024 and the stakes have never been higher for verifying credible information. People around the world have seen how deepfakes — synthetic media that leverage AI to replace one object with another — can deceive audiences and influence votes.

These AI techniques can produce text, images, videos, or audio recordings that seem real, but are artificially generated from public data. Consider these examples:

• Video deepfake: A finance worker at a multinational firm in Hong Kong was duped into sending $25 million to attackers who used AI-based deepfake video-cloning to create a multi-person “video conference” where everyone was fake, including the company’s CFO who supposedly authorized the transaction.
• Audio deepfake: A hacker successfully breached Retool’s multi-factor authentication (MFA) process and stole granular customer data by replicating the voice of an IT employee and getting another employee to divulge the MFA security token, resulting in access to the sensitive data.
• Audio deepfake: An attacker used recordings of Wiz CEO and Co-Founder Assaf Rappaport speaking at a conference to create a deepfake voice message that asked employees for their credentials.

Sure, manipulated media is nothing new (Photoshop, anyone?), but the generative AI models available today have taken the sophistication to a new level, significantly lowering the barrier to creating convincing deepfakes. And the threat doesn’t stop at politics. Celebrities, brands, and enterprises are also common targets.

We’ve had recent discussions with business leaders who told us they contended with deepfake attacks connected to major company events such as IPOs, M&As, product releases, and earnings calls—posing substantial risks to their businesses.

As investors in innovative security and trust startups, including current portfolio companies Heeler, Uptycs, Veza, and ActiveFence, we are following this nascent market closely. We hold ongoing conversations with security executives, as well as founders of deepfake security companies. In this blog, we tackle key questions such as: Should companies be adding deepfake security tools to their technology stacks? How strong is the market demand for deepfake security software? Which deepfake detection solutions and use cases are most mature and commercially viable today?

Deepfake Attack Vectors and Related Risks Expand

Conceptually, deepfakes are a more sophisticated type of phishing attack, designed to deceive recipients or audiences and prompt a specific action or reaction. While many companies have protocols in place to detect and prevent email- and text-based phishing attacks, they haven’t yet deployed the same safeguards for deepfakes and their varied modalities. Detecting deepfakes presents a new challenge: engaging and seemingly authentic content that’s much harder to ignore than an email with a suspicious link.

Not all deepfake attacks have business or monetary gain as their goals, either. Taylor Swift was the subject of a malicious deepfake text-to-image campaign that gained traction on X (Twitter).

These attacks underscore the significant risks deepfakes can pose to enterprises and individuals. Gartner identified at least five:

1. Stolen intellectual property and brand identity
2. Brand reputation damage in customer-facing content
3. Proliferation of disinformation
4. Fraud in identity verification
5. Monetary loss

As malicious deepfake attacks generate headlines and fear, companies and individuals will look for ways to detect and neutralize them. Accordingly, the legislative landscape and commercial market for deepfake security are evolving quickly in response.

Industry and Legislative Attempts to Reign in Deepfakes

The tech industry and state governments are spearheading ways to address deepfakes. Industry coalitions are setting standards for verifying the origins of digital content. At the same time, state lawmakers are attempting to legislate deepfake activity.

A number of notable tech companies have collaborated to form the Coalition for Context Provenance & Authenticity (C2PA), an initiative to develop technical standards for certifying the source and history of media content. The C2PA framework provides a helpful foundation, but its efficacy depends on widespread adoption by content creators, platforms, and consumers.

Several states have also enacted laws to criminalize the malicious use of specific categories of deepfakes. Although they vary somewhat, California and Texas laws allow victims of sexually explicit deepfakes to sue the creators. These states also allow political candidates to take legal action if deepfake videos are distributed close to elections. Although clearly on the radar of lawmakers, officials still need to figure out whether the US needs a federal law to deliver more consistency and clarity about deepfake activity.

The Market for Deepfake Security Tools Is Evolving Quickly

Mitigating the impact of deepfakes could eventually require organizations to deploy an array of security measures. Although most companies are not yet incorporating deepfake detection and mitigation tools into their technology architecture, that could change.

When we reach that tipping point, most will focus on solutions across the six categories outlined in Figure 1.

Figure 1. The 6 Categories of Deepfake Security Tools for Enterprises

Companies are already providing specialized solutions in each of these areas. Figure 2 identifies some category leaders in the current deepfake security market.

Figure 2. Deepfake & Brand Security Market Map

Some of these categories are more mature than others, presenting different levels of commercial potential.

Deepfake Image, Video, and Audio Detection

Deepfake image, video, and audio detection tools have broad application in both the public and private sectors. Existing use cases in call centers, HR, brand protection, account recovery, and fraud prevention bode well for their long-term viability.

They’re a hot topic among cybersecurity leaders as well, as evidenced by Reality Defender winning the RSA Sandbox competition earlier this year. However, deepfake detection is a capital-intensive endeavor, requiring heavy investments in R&D to keep ahead of fraudsters. It’s the typical cat-and-mouse game of cybersecurity: attackers continuously refine their deepfake models for more sophisticated attacks and the detection tools have to scramble to keep up.

Identity Verification

Similarly, identity verification is a necessary capability for deepfake detection, but most enterprises are already using a combination of tools to achieve adequate protection. Legacy solutions such as Nuance (voice biometrics) and Pindrop (voice and behavioral analysis) are well established.

Content Watermarking

Content watermarking makes intuitive sense for commercially licensed AI models, but open source models could be used freely without application of watermarks. Long-term success may be contingent on the development and acceptance of standards like C2PA by creators and AI-creation toolmakers alike.

Brand Protection and Content Licensing

Companies that provide brand protection and content licensing tools benefit from having customers (celebrities and brands) who are highly motivated to protect their images, likenesses, and reputations. This would be akin to the protections provided under the Digital Millennium Copyright Act (DMCA) of 1998.

An automated takedown feature meets a clear customer need, and sales cycles are relatively short. Providers of these solutions also have a built-in growth opportunity if they offer licensing features that allow celebrities or brands to monetize unique content.

Narrative Attack Protection

Narrative attack protection is becoming more important as disinformation campaigns are deployed for commercial gain. These AI-driven tools monitor and analyze large amounts of data, detect false narratives, and generate insights into how to counteract them. Banks, healthcare companies, and even advertising agencies have all deployed solutions in this area. However, a critical hurdle for providers is demonstrating adequate ROI to justify ongoing adoption.

What’s Next for Deepfake Security

Deepfake technology has greatly accelerated over the past few years, and the advent of generative AI tools has only made it easier to create and distribute synthetic content. The potential for deepfake attacks poses considerable risk. However, our observation of the current market is that companies tend to look for deepfake security solutions after they have been attacked when they face pressure from the board or executives to implement solutions quickly.

With annual contracts reaching upwards of $250,000, will enterprise customers proactively seek out these tools? Will media coverage of corporate deepfake attacks—which are sure to come—raise awareness and demand from businesses who want to keep their brands out of those headlines?

As the deepfake security landscape continues to evolve, we will continue to assess its potential, identifying which specific verticals and enterprise use cases are most compelling to the market. Innovative solution providers that can differentiate themselves from more basic, reactive competitors will emerge as leaders in the space.

We believe deepfake security tools are poised to become a part of the enterprise security stack as attacks progress in sophistication. In the interim, we see leaders of targeted functional teams such as HR and customer support adopting these solutions and championing them internally.

If you’re already building one of those solutions or you just have thoughts on our perspective, reach out to Dave Zilberman or Bayan Alizadeh.