Market for synthetic media: Generating value for AI-generated content

• Author:
• Author name

  Quantumrun Foresight
• March 23, 2022

Insight summary

Synthetic media, particularly deepfakes, is a rapidly evolving technology that can be used to alter videos and misrepresent a person's actions or words. From its early stages as a hobbyist's tool to its current underground market, the technology's applications range from artistic creation to potential criminal impersonation and political manipulation. The long-term implications of this trend include shifts in cultural norms, legal frameworks, advertising strategies, and even educational systems.

Market for synthetic media context

‘Synthetic media’ refers to the production of data or media using AI, typically to mislead people or change the meaning of a piece of media. Given the applications, synthetic media may see increased use for criminal or political purposes. The most well-known form of synthetic media are ‘deepfakes,’ where a creator takes a video and edits a person’s face to replace who appears to be in the video. This technology can misrepresent a person’s participation in an act that is captured on camera.

An intelligence firm, Deeptrace, has projected that the technological development and consumption of synthetic media has developed over three phases. The first phase of development was the time when the technology was still new and unregulated because it was primarily limited to hobbyists. This stage has since passed. 

The second phase, happening now (2020 to 2025), is when the technology becomes widespread enough that an underground market emerges to meet supply and demand. Given the questionable legal nature of synthetic media, this market is presently underground and only semi-regulated. Current synthetic media products are mostly pornographic, while parodic works that appear as special filters on social platforms like Instagram, Snapchat, and TikTok form a minority of such creations.

Disruptive impact

The development of synthetic media technology, particularly deepfakes, has implications that reach far beyond current applications. As the technology matures, it may become a powerful tool for criminal impersonation and nation-state activities. For individuals, the potential misuse of deepfakes could lead to personal and financial harm.

Imagine a scenario where a deepfake video convincingly portrays a person admitting to a crime they did not commit or engaging in other damaging behavior. This trend could lead to reputation damage, legal issues, or even personal safety concerns. Individuals may need to become more vigilant and aware of the content they consume, understanding that seeing is no longer necessarily believing.

Companies may face challenges related to intellectual property theft, fraud, and corporate espionage. A well-crafted deepfake could be used to impersonate a CEO in a video conference, leading to the release of sensitive information. Governments may need to take a more active role in regulating the production and dissemination of deepfakes, balancing the need for innovation and free speech with the potential risks to security and privacy. The existing laws targeting deepfake pornography and political uses are just the beginning, and a more comprehensive legal framework may need to be developed to address the broader risks and ethical considerations of this emerging technology.

Implications of the synthetic media market

Wider implications of the synthetic media market may include:

• Funding the continued development of synthetic media technology, leading to the emergence of new artistic forms and entertainment genres, as well as making synthetic media an attractive niche for content creators.
• Growing the underground access to synthetic media creators, leading to a potential legitimization of the niche over time, which may influence the way content is consumed and regulated.
• Fostering a greater acceptance of deepfakes for both pornographic and parodic uses, leading to shifts in cultural norms and potentially challenging existing legal frameworks around obscenity and satire.
• Encouraging the use of deepfakes for impersonation or fraudulent purposes, leading to increased security measures and vigilance in both personal and professional communication.
• The development of new tools and methods to detect and combat deepfakes, leading to a continuous arms race between creators and detectors, impacting the technology industry and law enforcement.
• A shift in advertising and marketing strategies, utilizing synthetic media to create highly personalized content, leading to more targeted and effective campaigns but also raising ethical concerns about manipulation and consent.
• The integration of synthetic media into educational systems, leading to enhanced learning experiences but also requiring new guidelines to ensure accuracy and ethical use.
• The potential use of deepfakes in political campaigns, leading to more stringent regulations and public discourse around authenticity, transparency, and the role of technology in democracy.

Questions to consider

• How quickly do you believe deepfake technology will advance to the point that believable deepfakes are within reach?
• How tight should the legislation be around synthetic media? Or how should such media be responsibly regulated?