A social media-based artificial intelligence chatbot has been suspended in South Korea after stirring a series of controversies with discriminatory and vulgar language, leaving lessons for all parties involved in the related sectors dealing with AI.

The chatbot Lee Luda, developed by local startup firm Scatter Lab, was introduced last December to run on Facebook Messenger. Designed to mimic a 20-year-old woman, it has attracted more than 400,000 users via Facebook so far.

It, however, became a subject of controversy after some online users shared ways to “educate” the chatbot with sexually abusive language against women and discriminatory comments on minorities on forums.

The service’s machine learning architecture was fed with users’ interactions, including expressions that were homophobic or discriminative.

The magnitude of the issue also escalated as the majority of its users were teenagers.

According to Scatter Lab, teenagers account for around 85 percent of its total user base, while those users in their 20s accounted for around 12 percent.

“The latest controversy with Luda is an ethical issue that happened due to the lack of awareness about the importance of ethics in dealing with AI,” said Jeon Chang-bae, the head of the Korea Artificial Intelligence Ethics Association, a non-government organization that researches AI ethics.

To prevent such issues, the developer should have developed a separate AI that can censor language datasets. But, this is currently not possible due to technical limits, Jeon said.

“For the moment, it is important for both developers and users to note the ethical guidelines, when launching and using AI-based services,” Jeon added.

The issue with Luda resembles the infamous case of Microsoft’s Tay, an AI Twitter bot that speaks like a teen, he noted.

In 2016, Microsoft shut down Tay within 16 hours, after it turned into a brazen anti-Semitic.

Jeon stressed that the latest incident should be a wakeup call for the country to recognize the need for greater public engagement and education with regards to the potential dangers in dealing with AI.

“Students tend to get easily addicted to such things or swayed by opinions from their close friends,” a teacher at an elementary school in Seoul said.

“The latest issue could have been a result of sharing indiscreet and unethical thoughts via online communities. Yet, the issue could also be a reflection of a failure of both home discipline and public education, which were unable to offer sexual and political values that students could have gone by otherwise,” she added.

Meanwhile, some are questioning how Scatter Lab was able to secure 10 billion KakaoTalk messages for its “Science of Love” app. Scatter Lab gained attention in the industry with the application that analyzes the degree of affection between partners by submitting actual KakaoTalk conversations.

The Personal Information Protection Commission and Korea Internet & Security Agency of South Korea announced Monday that they would investigate whether Scatter Lab violated any information privacy laws.

By Shim Woo-hyun (ws@heraldcorp.com)