(Original title: "Science" magazine: Artificial intelligence also learned racial prejudice when acquiring human languages) Microsoft's artificial intelligence (AI) chat robot Tay 澎湃 News Trainee Reporter Han Hanqi Reporter Jiang Chenyue In March of last year, Microsoft’s artificial intelligence (AI) chat robot Tay (U.S. sister version of Xiao Bing) was on Twitter and could chat with any of her Twitter users. Just 24 hours later, a sweet, courteous “little girl†started to swear, even bursting with racist and sexist remarks. This quick "AI Hitler" caused netizens to be shocked and was rushed by Microsoft into the "small dark house." In the April 14 issue of Science, a joint research team from Princeton University and the University of Bath in the United Kingdom published a new study confirming this phenomenon: AI also displays various prejudices and discrimination. These prejudices involve Race, gender, age, etc. One of the authors of the paper, Professor Joanna Bryson of the University of Bath in the United Kingdom, said, “People will say that experiments show that AI is discriminatory. No. This actually means that we humans have discrimination and they have been learned by AI.†The dark side of deep learning The team invented a word embedded correlation test (WEAT) method to test some of the inherent biases of AI. The researchers' inspiration comes from an implicit association test (IAT) psychological tool. In the IAT test, some characteristic vocabulary flashed on the computer screen, and the different response speeds of the subjects to these vocabulary combinations hinted at some associations hidden in people's hearts. If the subject responds more quickly to phrases that consist of white common English names and phrases with positive vocabularies, black common English names, and negative vocabularies, this points to racial prejudices that may exist in the subconscious. In the WEAT test, researchers no longer test the speed at which AI responds to different combinations of words but quantify the degree of similarity between the AI's embedded vocabulary. Word Embedding is an important tool in machine learning. It is mainly used in text analysis, web search, and translation. Another author of the paper, Arvind Narayanan of Princeton University, said, "The reason why we chose to analyze vocabulary inlays is that it has greatly helped AI understand our language in the past few years." Specifically, the vocabulary of the human language is entered into the computer in the form of a series of numbers. Researchers measure the meaning of a word in different dimensions and assign a number to form a series of numbers. The numbers are mainly based on other words frequently associated with this term. This seemingly mechanical, pure mathematics approach helps the machine understand the rich and complex semantics of humans more than simple semantic definitions. By analyzing which vocabulary strings are closer, the researcher knows what associations the AI ​​is embedded with. For example, "ice" and "water" will often appear together in the text and their strings will be more similar. However, in addition to these simple associations, researchers have discovered that AI is also embedded with more complex associations. For example, "women" and "women" will have closer professional contacts with some arts and humanities, while "male" and "men" will have more professional connections with mathematics and engineering. In addition, similar to the performance of many British and Americans in IAT testing, AI prefers to associate white English names with positive vocabularies such as “talent†and “happiness,†while associating black English names with some unpleasant vocabulary. Associations. Deep-learning based on a large number of online texts has led to the rapid development of smart translations including Google Translate. But now it seems that AI has also been “forced†to learn some of the prejudices inherent in human culture in the process of learning human languages. AI watchdog The study used a database called "Common Crawal" which contained 844 billion words from the public materials on the Internet. Researchers also got similar results when they experimented with Google News data alone. Sandra Wachter, a researcher in data ethics at Harvard University, said, "The world is biased and historical data is biased, so it's not surprising at all that we get biased data." Research shows that the existing inequality and prejudice structure in human society will be further aggravated by emerging technologies. Given that humans are devolving everyday life and work step by step to AI, this exacerbation is dangerous. This risk is further magnified by another factor: Humans have at least moral constraints, and when programmers develop AI, they do not think of configuring them with an ethical and moral algorithm to resist discrimination in the input data. However, Wachter also mentioned that people do not have to treat it as a threat. "At least we can know that algorithms are biased. Algorithms won't lie to us. But humans can lie," she said. According to Wachter, the real question is how to eliminate algorithmic biases without depriving AI of human language comprehension. In fact, AI can also be a human "watchdog." "Human beings can theoretically develop a system to detect prejudices in human decision-making and take some measures. This is complicated, but we should not escape this responsibility," said Wachter. Do you have language or discrimination first? This study has other meanings. For a long time, people have disputed the origin of discrimination in human languages. One party believes that the discriminatory tendencies in the human text will affect the social value ethics subtly; the other party believes that the discriminatory value in human society leads to the emergence of discriminatory content in the text. Now, WEAT is expected to answer this question similar to the previous chicken or egg. WEAT can test the AI ​​against a text in a historical phase and compare it with the IAT test results in a later historical phase. If changes in WEAT results precede changes in IAT results, this suggests that human language catalyzes the formation of discriminatory views in society; on the contrary, it indicates that it is a discriminatory value in the subconscious of human beings, which in turn generates texts with embedded discrimination. .