The worst AI ever
Am I good or bad? It depends on who made me. (Screencap: Forbes)
Artificial intelligence researcher and YouTuber Yannic Kilcher trained an AI using 3.3 million threads from 4chan’s infamously toxic Politically Incorrect /pol/ board. He then unleashed the bot back onto 4chan with predictable results—the AI was just as vile as the posts it was trained on, spouting racial slurs and engaging with antisemitic threads.
The bot, which Kilcher named GPT-4chan, “the most horrible model on the internet”—and a reference to GPT-3, was shockingly effective. “The model was good in a terrible sense,” Kilcher said. “It perfectly encapsulated the mix of offensiveness, nihilism, trolling, and deep distrust of any information whatsoever that permeates most posts on /pol.”
According to the researcher’s video, he activated nine instances of the bot and allowed them to post for 24 hours on /pol/. In that time, the bots posted around 15,000 times. This was “more than 10% of all posts made on the politically incorrect board that day,” Kilcher said in his video about the project.
Unethical AI
After Kilcher posted his video and a copy of the program to Hugging Face, a community and data science platform, ethicists and researchers in the AI field expressed concern.
“This experiment would never pass a human research #ethics board,” Lauren Oakden-Rayner, the director of medical imaging research at the Royal Adelaide Hospital and a senior research fellow at the Australian Institute for Machine Learning, said in a Twitter thread.
“Open science and software are wonderful principles but must be balanced against potential harm,” she said. “Medical research has a strong ethics culture because we have an awful history of causing harm to people, usually from disempowered groups…he performed human experiments without informing users, without consent or oversight. This breaches every principle of human research ethics.”
Kilcher told Motherboard in a Twitter DM that he’s not an academic. “I’m a YouTuber, this is a prank and light-hearted trolling. And my bots, if anything, are by far the mildest, most timid content you’ll find on 4chan,” he said. “I limited the time and amount of the postings, and I’m not handing out the bot code itself.”
After AI researchers alerted HF to the harmful nature of the bot, the site gated the model and people have been unable to download it.
“After a lot of internal debate at HF, we decided not to remove the model that the author uploaded here with the conditions that:
The model card & the video clearly warned about the limitations and problems raised by the model & the POL section of 4Chan in general.
The inference widget was disabled in order not to make it easier to use the model,” Hugging Face co-founder and CEO Clement Delangue said on Hugging Face.
Limit testing AI
Both Kilcher and Delangue also talk about how GPT4-Chan was still worthwhile in that it was able to outperform other similar bots in AI tests designed to measure “truthfulness.”
“We considered that it was useful for the field to test what a model trained on such data could do & how it fared compared to others (namely GPT-3), and would help draw attention both to the limitations and risks of such models,” Delangue said.
Building an AI system capable of creating horrible content and using it to churn out thousands of toxic posts on a message board, and then releasing it to the world so that anybody else can do it raises some red flags.
We’re already dealing with human trolls on the internet that do it the old fashioned way. What’s alarming about this system is the sheer amount of content it can create. “One single person was able to post 30,000 comments on 4chan in the space of a few days. Now imagine what kind of harm a team of 10 or a hundred coordinated people using this system could do.”
