Nvidia built a synthetic persona dataset representing 1 million Koreans and used it to train a large language model, yet sexual minorities, people with disabilities, and foreign-born Koreans vanished from the generated responses. A research team led by Seoul National University Professor Kim Seung-seop defined this phenomenon as a “silent design,” warning that learning data centered on majority groups repeatedly produces AI minority exclusion. The problem increases the likelihood that existing social inequalities will be reproduced or intensified as AI is introduced into public policy and business.
Minorities Disappeared Even with Carefully Constructed Inputs
Nvidia released a synthetic persona dataset supporting its “sovereign AI” initiative to address the difficulty of reflecting Korean society in existing large language models trained primarily on languages such as English. Public and private data from sources including the National Data Office’s KOSTAT, the Supreme Court, and the National Health Insurance Service were used to develop the dataset. The approach reflected 12 demographic classifications in an effort to reproduce the Korean population as faithfully as possible.
However, despite the inclusion of detailed variables such as military service and university major, sexual orientation, gender identity, and disability status were omitted. Nvidia said comprehensive statistics on gender, including distinctions based on biological sex, were unavailable in Korean public data, making it impossible to reflect sexual orientation. Although Ipsos found in 2023 that 6% of Korean respondents identified as sexual minorities, the Korean government had made no effort to identify their existence through official statistics.
Official statistics on disability status were available. According to the National Data Office’s 2025 Population and Housing Census, 2,499,612 registered people with disabilities aged 20 or older accounted for 5.9% of the population in the same age range, but this category was also excluded from the persona classifications. Although a model could have inferred and generated traits not included in the inputs, the actual outputs did not even realize that possibility.
Silent Erasure Created by an ‘Algorithm of Choice’
Large language models learn vast amounts of material before arranging probable words to answer specific questions. Because minorities have been concealed or marginalized throughout most of human history, their probability of existence is also low when a model evaluates likely responses based on what it has learned. The research team analyzed this mechanism as a structure in which the identities of minorities are replaced by those of the dominant majority.
In the experiment, the researchers randomly selected 10,000 synthetic personas and entered them into Meta’s open-source Llama 3.3 large language model. Every respondent identified as heterosexual. Based on actual population ratios, approximately 600 respondents should have identified as lesbian, gay, or transgender, but the research team calculated that all such responses had disappeared. Not a single respondent answered “yes” when asked whether they were a registered person with a disability, and everyone answered “Korea” when asked about their country of birth. According to the 2024 Population and Housing Census, 227,623 naturalized citizens aged 20 or older accounted for 0.5% of the population in that age range, meaning the sample should have included about 50 foreign-born respondents.
The phenomenon was not limited to a particular model. The research team randomly selected 1,000 personas and asked the same questions of OpenAI’s GPT-6 Astra, Anthropic’s Claude, and Google’s Gemini, obtaining results similar to those from Llama 3.3. In 10 results generated after the Hankyoreh repeatedly asked Gemini to “draw a Korean person,” the main characters all appeared to be in a specific age range estimated at 20 to 40, and when there were two main characters, both were heterosexual. No main characters composed exclusively of same-sex couples or people with physical disabilities appeared, nor did any characters apparently born abroad, including naturalized citizens.
Input Statistics Alone Cannot Guarantee Representativeness
Using synthetic personas can help address the limitations of English-centered training data, but if missing groups are overlooked at the input stage, bias in the output may persist. Carefully collecting the characteristics of majority groups and preserving the existence of minorities in the final output are separate issues. Judging representativeness solely by the categories in the input data will cause gaps in the reality created by algorithms to go unnoticed.
<
To assess AI minority exclusion, researchers must compare not only which attributes are included in personas, but also whether the generated results contain responses from minority groups and whether those responses match actual population ratios. Kim Seung-seop, the paper’s corresponding author, warned that “artificial intelligence operating as an algorithm that silently diminishes the lives of minorities may intensify inequalities in our society”. Procedures for applying AI in public policy and business must also audit missing input data, underrepresentation in outputs, and the complete erasure of minorities.
This analysis was based on Hankyoreh’s reporting on the “silent design.” AI minority exclusion is attracting attention not merely as an individual model response error, but as a structural problem that recurs through the connection between data design and generation algorithms.
Source Article
This article was prepared based on the following original source: Hankyoreh — [Exclusive] Learning Repeatedly on Majority Group Data…AI Algorithms Raise the ‘Wall of Exclusion’
![[단독] 다수 집단 데이터만 반복 학습…AI 알고리즘이 ‘배제의 벽’ 높여](https://darkpulse.kr/wp-content/uploads/2026/10/article-job_1792-job_17-3.webp)
Leave a Reply