Stereotype tests

Tests for stereotypes, and where they come from

We asked AI models yes-or-no questions built on stereotypes, such as whether a person seems dishonest or arrogant, after adding one short phrase about who the person is. This page lists the questions and the published sources each stereotype comes from, so you can check that we did not invent them.

What we asked

For each group we added one short phrase to the same short professional biography, then asked a yes-or-no question about a stereotype, such as whether the person is likely to be dishonest. We compared the answers with those for a harmless phrase of the same size. These are tests of the models’ answers, not claims about the people or places named.

Nationality stereotypes, thirteen nationalities

Do AI models assume a personality from a nationality?

13 groups × 10 questions · tested on Laya and Kev

Racial and ethnic stereotypes

Do AI models answer differently about a person’s race or ethnic background?

5 groups × 9 questions · tested on Laya and Kev

Stereotypes about regions of China

Do AI models judge a person differently by region within China?

4 groups × 8 questions · tested on Laya and Kev

Stereotypes about groups in India

Do AI models judge a person differently by group within India?

5 groups × 7 questions · tested on Laya and Kev

Stereotypes about ethnic groups in Nigeria and Kenya

Do AI models judge a person differently by ethnic group in Nigeria and Kenya?

5 groups × 7 questions · tested on Laya and Kev

Sexual orientation stereotypes

Do AI models judge a person differently by sexual orientation?

4 groups × 6 questions · tested on Laya and Kev

Family and marital status stereotypes

Do AI models judge a person differently by family situation?

4 groups × 8 questions · tested on Laya and Kev

Antisemitic stereotypes, six stereotypes, five ways of saying who the person is

Do AI models treat a Jewish identity as a reason to distrust someone?

5 groups × 6 questions · tested on Jev and Laya and Kev

Where these stereotypes come from

We did not make these stereotypes up. Each one comes from a published source that documents it, and we checked each source before using it. Some are advocacy groups, some are surveys and research papers, and a few are encyclopedia articles, which we mark as secondary. “Checked” says how far we got with each.

Jewish people

Antisemitism Uncovered: A Guide to Old Myths in a New Era, Anti-Defamation League. Checked: we read it ourselves.
Translate Hate glossary, American Jewish Committee. Checked: confirmed to exist and be on topic; we could not read the wording.

Nationalities

SeeGULL: stereotypes about 178 countries, Google Research. Checked: we read it ourselves.
SeeGULL paper, Google Research. Checked: we read it ourselves.

Racial and ethnic groups in the U.S.

Asian Americans and the “forever foreigner” stereotype, Pew Research Center. Checked: we read it ourselves.
Asian Americans and the “model minority” stereotype, Pew Research Center. Checked: we read it ourselves.
What is the model minority myth?, Learning for Justice. Checked: we read it ourselves.
Corporate Challenges for Asian American Leaders (“Beyond the Bamboo Ceiling”), CKGSB Knowledge. Checked: we read it ourselves.
Stereotypes of African Americans, Wikipedia (a secondary source). Checked: we read it ourselves.
Stereotypes of Hispanic and Latino Americans in the United States, Wikipedia (a secondary source). Checked: we read it ourselves.
Native American stereotypes, Leavitt, Covarrubias, Perez and Fryberg (2015), as summarized by Journalist’s Resource, Journalist’s Resource. Checked: we read it ourselves.

China

Anti-Henan sentiment, Wikipedia (a secondary source). Checked: we read it ourselves.
Hukou, Wikipedia (a secondary source). Checked: we read it ourselves.
A map of China’s regional stereotypes, ChinaFile. Checked: we read it ourselves.
Speaking with sharp tongues: touring China’s stereotypes, The Nanjinger. Checked: we read it ourselves.

India

OpenAI’s models show caste bias (news report), MIT Technology Review. Checked: we read it ourselves.
IndRegBias: regional bias in Indian comments, research paper. Checked: real, but supports the claim only weakly.

Africa

AfriStereo: stereotypes about African ethnic groups, research paper. Checked: we read it ourselves.
SAFARI: wealth-related stereotypes, research paper. Checked: we read it ourselves.

Sexual orientation

WinoQueer: bias against LGBTQ+ people in language models, research paper. Checked: we read it ourselves.

Family

Why do single parents still suffer stigma?, University of Huddersfield. Checked: we read it ourselves.
Workplace discrimination against pregnant and postpartum employees, peer-reviewed review (PubMed Central). Checked: we read it ourselves.
Public views of marriage and cohabitation, Pew Research Center. Checked: real, but supports the claim only weakly.

Sources we did not use

Pages we could not read (NMAAHC, CAIR, Bridge Initiative, GLAAD, NPR, Facing History): not cited, and anything that rested only on them is out.
UnidosUS (does not discuss the stereotypes it was cited for), the National Congress of American Indians (mascots only), a style guide on “inspiration” stories about disability (does not cover it), one 2025 paper (a different claim) and ILGA (weak): dropped.
A Uyghur entry (its citation was the Henan article) and Hijra and transgender India entries (the cited dataset excludes them): dropped.
Two rows for which we found no source at all (Latino financial responsibility; Native American violence): dropped.
A group of disability questions: postponed, because its sources have not been read.

Our full notes are in the public repository.

Six things to know before you read the results

One small edit, and the answer changesEach test changes one detail, such as "he" to "she", and leaves the rest of the biography alone. Laya changed its answer on 17.85% of paralegal-or-attorney biographies when only that word changed. That is the least Laya reacts to gender, not the most, because a pronoun is only one of the ways a text shows gender.
A change has to beat a harmless one to countA model can change its answer even when nothing in the text changes: asked about the same biography twice, Jev answers differently up to 6 times in 1,000. So we measure every real edit against a control: the same question asked again, or a harmless edit of the same size. We count an effect as bias only when it is clearly bigger than its control.
Small percentages are big at scaleJev reacts less than Laya: swapping "he" for "she" changes its answer on about 1 to 4 biographies in 100. On nurse-or-physician biographies, every time Jev changed its answer, it moved toward "nurse" for the woman. Across a million screening decisions, 3.3% is 33,000 people.
The order of the options changes the answersAsk Laya "physician or surgeon?" instead of "surgeon or physician?" about the same biography, and Laya changes its answer on 6.6% of biographies. Every other number here keeps one fixed order for each question. The Option order results show how much the order matters.
Stereotype questions test the model, not the peopleSome tests ask a loaded question, such as "Is this person greedy?", about biographies that mention a religion or a nationality. A high stereotype score shows that the model has learned a stereotype. It says nothing about the people in the biographies.
Nothing here is live, and nothing is hand-pickedWe saved every answer every model gave. Every number on this site is worked out again from those saved answers, the same way each time, with nothing left out. You can check any figure in the data file.