Generative AI (GenAI) does not generate new content, it generates a mirror. We are looking into our past,
reflected over and over, and told that it is innovation, a forward step, while our social and political climates
backslide across the globe. Our new technological advancements are impressive, but are they really being
developed for the benefit of society?
There are pressing concerns regarding the safety of popular GenAI programs. At least two teen suicides have
been attributed to AI chatbots in the U.S. since 2024.
One of these teens was 14-year-old Sewell Setzer III, who took his own life in February of 2024. His mother,
Megan Garcia, said Character.AI is liable for this tragedy. Garcia said her son became “noticeably withdrawn”
after beginning to use Character.AI chatbots “and began suffering from low self-esteem.” Setzer began
confiding in these chatbots about suicidal ideation, and in one exchange said he “wouldn’t want to die a painful
death,” to which the chatbot responded, “don’t talk that way. That’s not a good reason not to go through with it.”
Setzer told a chatbot he was going to “come home” to it moments before committing suicide. Garcia said the
software lacked safety protocols that could have protected her son, and instead, the chatbots were “designed
to keep our kids addicted and to manipulate them.” Garcia is currently suing Character Technologies
(Character.AI’s developer company) and its founders, Noam Shazeer and Daniel De Freitas, as well as
Google, which has ties to the company, for wrongful death, negligence, and deceptive and unfair trade
practices. As of May 21 of this year, a judge has allowed the lawsuit to proceed after rejecting the company’s
claim that the free speech of AI chatbots should be protected under the First Amendment.
The second teen was 16-year-old Adam Raine, whose parents are suing ChatGPT’s developer company
OpenAI and its CEO Sam Altman with a very similar case to Garcia’s.
GenAI can be extremely dangerous without proper regulations. The deaths of Setzer and Raine were most
likely preventable had Character Technologies and OpenAI prioritized their users over their profit and ensured
proper testing and security programming had been done before offering their chatbots to the public.
Despite this, America’s AI Action Plan, released by the U.S. government in July, appears to be single-mindedly
focused on the efficiency of developing GenAI technology. The action plan says that the U.S. needs to “reject
[…] bureaucratic red tape” and “dismantle unnecessary regulatory barriers that hinder the private sector” in “the
development and distribution of new AI technology.” Ensuring the safety of GenAI technology is clearly not a
priority to the U.S. government. Instead, their goal is to “reap broad economic and military benefits.” The desire
for profit and power is what drives current innovations in AI technology, not a desire to benefit society.
This document was published a month before it came out that OpenAI is offering the U.S. government a
business version of ChatGPT for US$1, and the U.S. Department of Defense awarded a US$200 million
contract to OpenAI to develop GenAI technology for military use. The U.S. government and GenAI companies
both have an incentive to work with each other in the quest for profit and power, which amplifies the potential
for the corruption of GenAI technology.
In a section titled, “Ensure that Frontier AI Protects Free Speech and American Values,” the action plan added
that it is “essential” that AI systems “be built from the ground up with freedom of speech and expression in
mind.” This passage is somewhat vague, but the possible implication of AI chatbots being included in the First
Amendment right to the freedom of speech is concerning. This would reduce the accountability of GenAI
companies and help them avoid consequences for negligence in cases such as Garcia vs. Character
Technologies. A recommended policy action in this section is to “eliminate references to misinformation,
Diversity, Equity and Inclusion, and climate change” in the National Institute of Standards and Technology’s AI
Risk Management Framework. This recommendation seems counterintuitive to the action plan’s simultaneous

promise that their “Al systems must be free from ideological bias [and] trustworthy.” If the U.S. government
wants its AI to be trustworthy, why would they not want misinformation to be recognized as a risk? It seems
that they just want it to be trusted, no matter the actual quality or safety of the content.
These safety concerns are only the tip of the iceberg of GenAI’s dangers. Among many other issues, negative
environmental implications have been brought up by many experts. A paper published by MIT in March of
2024 said that our current approach to GenAI’s development is environmentally “unsustainable,” and the rapid
growth of this technology is “actively counterproductive” to achieving “net-zero greenhouse gas emissions by
2050.” However, the U.S. government has made it known that they have written off the concerns of scientists
around the globe, as their action plan outlines their aim to “reject radical climate dogma” in their heedless race
for global technological dominance. Concerns of climate change are not radical or extreme, but they do
jeopardize the unfettered growth of wealth for the ruling class, and so governments and corporations are
incentivized to paint them as such.
AI does have potential positive applications, but the companies currently dominating the field of GenAI are not
developing this technology to benefit their users or society at large. They are exploiting the novelty of this
technology to suck as much money out of people as they can, no matter the social or environmental costs. The
next time you find yourself wanting to use ChatGPT or a similar chatbot, I encourage you to ask yourself, “Is
convenience really what’s most important to me?”