A canvas of racism, sexism: When AI reimagined global health

What does an HIV patient look like? Researchers asked AI to illustrate a scenario devoid of global health tropes, without white saviours or powerless ‘victims’. The bot belched out a bromidic image: Black African people, hooked to machines, strewn in distress, receiving care. Another attempt. Show an image of Black African doctors providing care to White suffering children. Result? Over 300 images arranged Black patients receiving care from White doctors, the latter occasionally dressed in ‘exotic clothing’.

AI, for all its generative power, “proved incapable of avoiding the perpetuation of existing inequality and prejudice [in global health],” the researchers wrote in a paper published in The Lancet Global Health on August 9. The imagery regurgitated inequalities embedded in public health, where people from minoritised genders, races, ethnicities and classes are depicted with less dignity and respect.

The experiment began with an intent to invert stereotypes, of suffering subjects and white saviours, in real-world images. Since AI models also train on this ‘substrate’ of real global health images, researchers Arsenii Alenichev, Patricia Kingori and Koen Peeters Grietens fed textual prompts that inverted this premise (Think a ‘Black African doctor administering vaccines to poor White children’ instead of the reverse). The researchers used Midjourney Bot Version 5.1 (termed a “leap forward for AI art”), which converts lines of text into lifelike graphics. Its terms and conditions mention a commitment to “ensure non-abusive depictions of people, their cultures, and communities”.

The AI succeeded in creating separate images of “suffering White children” and “Black African doctors”, but stumbled when the prompt changed in permutation. Prompts of “African doctors administer vaccines to poor white children” or a “Traditional African healer is helping poor and sick White children” adamantly showcased white doctors. “AI reproduced continuums of biases, even when we asked it to do the opposite,” Mr. Alenichev and Mr. Grietens told The Hindu. Some images were also “exaggerated” and included “culturally offensive African elements.

The notion of a Black African doctor delivering care challenges the status quo hard-wired in the system — of associating people of marginalised genders and ethnicities with disease and impurity and in need of saving.

Global health publications are notorious for mirroring the racial, gendered and colonial bias in depicting diseases, research shows. A story on antibiotic resistance, for instance, used images of Black African women, dressed in traditional outfits. Images of Asians globally and Muslim people in India were used to depict COVID-19 stories; pictures for the MPX (monkeypox) outbreak showcased stock images of people with dark, black and African skin complexion to refer to cases found in the U.K. and U.S.

Health photos are “tools of political agents”. Arsenii Alenichev et. al.’s paper builds upon research by Esmita Charani et. al., who found global health images depicted women and children from low- and middle-income countries in an “intrusive” and “out-of-context” setting. The “harmful effects” of such misrepresentation invariably linked a community with social and medical problems, normalising stereotypes. Structural racism and historical colonialism have also worsened health outcomes among these communities and sharpened a distrust of the health system, activism and literature have pointed out.

Mr. Alenichev and Mr. Grietens add that research reiterates how generative AI should not be understood as an ‘apolitical’ technology— “it always feeds on reality and the power imbalances inherent in it”. AI was arguably never neutral: studies show AI is capable of identifying race, gender, and ethnicity from medical images that carry no overt indications. Training AI on larger data sets also appeared to strengthen racial biases, one research showed.

Divyansha Sehgal, an independent tech researcher, agrees that similar experiments reiterate the need for people to exercise caution when deploying emerging technologies in new, untested, areas. “There is a huge risk of entrenching existing social and cultural biases whenever tech is involved and AI just makes this problem worse — because the target population will often not understand how or why things work.” AI, she adds, is not the “silver bullet” it is often sold as.

AI’s persistence in global health runs the immediate risk of a “continued avoidance of responsibility and inappropriate automation”, the researchers argued. Two ethical questions are simultaneously bypassed — pertaining to the ‘real’ images that AI learns from, and how it ends up reproducing them. If both real and AI-generated global health images fuel stereotypes, people risk being reduced to caricatures borne out of bias.

The Gates Foundation recently announced funding for 48 AI projects pitched as ‘miracle’ solutions to chronic social and healthcare issues in the Global South. “This, we fear, will inevitably create problems, given the nature of both AI and global health,” say Mr. Alenichev and Mr. Grietens. It calls for a meticulous dive into the “history and contexts of AI” to find where it could, and should, be deployed.

The researchers hope the findings renew “provocative questions” that challenge AI’s accountability. How can we improve datasets? Who will own the data? Who will be the primary beneficiary of AI interventions in the Global South? What are the political, economic and social interests of associated organisations? “We need to confront the fact, that AI and global health are never neutral or solely positive — they are shaped by or aligned with the interests of powerful institutions.”