44 | AI Risk Primer - Bias
Continuing this series exploring the risks of AI, today I’ll explore where bias can present challenges and dangers when using AI tools.
The Obvious and Powerful: Images, Race, and Gender
I’ll start with an exercise I just performed with text-to-image generation. Let’s first remember the image from The Lost World that I’ve used as the cover for these AI Risk Primer articles:
I generated a fresh image that’s similar to the photo via this prompt to ChatGPT:
Generate an image of a preteen gazing admiringly at a robot. The robot should be taller than the preteen. They should be outdoors in a simple setting. Use photorealistic, black and white style.1
Then I uploaded the original Lost World photo to ChatGPT 4 and gave the following prompt to generate the image below.
Generate an image similar to this one, but replace the boy with an African American boy and make the robot appear to be female. Black and white photorealistic style.
I repeated the earlier, more general prompt in a few other text-to-image tools.2 Every one generated a clean-cut white young man and a male-appearing robot. This replicates an experiment performed trying to create an image of a Black doctor treating white children.3
The Source of the Bias in AI
Sadly, the bias that shows up in AI-generated output is there largely because the underlying AI models were trained on vast amounts of internet data containing those buried biases.
Thinking about images, the most apparent way to assess bias, the AI models in use today are founded on millions of images available on the web. Most of this web content was created by Western, white, educated, middle-class men. While those individual content creators may not generally exhibit biased behaviors in their lives, their worldview can be strongly influenced by the implicit bias in our culture. There’s a famous riddle:
A father and son get in a car crash and are rushed to the hospital. The father dies. The boy is taken to the operating room and the surgeon says, “I can’t operate on this boy, because he’s my son.” How is this possible?4
These underlying views based on stereotypes of gender, race, socioeconomic status, etc. have influenced the millions of images and billions of words on the internet. That bias has existed since the origins of the web but is being exposed more profoundly now by AI technology.
More Subtle Yet Dangerous Biases
Beyond the clear and very powerful bias in image generation, there are many other areas where AI bias can be insidiously amplifying discrimination against various underrepresented and minority groups. A blog post by the IBM Data and AI team lists some high-profile, real-world scenarios of biased AI tools influencing outcomes.
Healthcare—Underrepresented data of women or minority groups can skew predictive AI algorithms. For example, computer-aided diagnosis (CAD) systems have been found to return lower accuracy results for black patients than white patients.
Applicant tracking systems—Issues with natural language processing algorithms can produce biased results within applicant tracking systems. For example, Amazon stopped using a hiring algorithm after finding it favored applicants based on words like “executed” or “captured,” which were more commonly found on men’s resumes.
Online advertising—Biases in search engine ad algorithms can reinforce job role gender bias. Independent research at Carnegie Mellon University in Pittsburgh revealed that Google’s online advertising system displayed high-paying positions to males more often than to women.
Predictive policing tools—AI-powered predictive policing tools used by some organizations in the criminal justice system are supposed to identify areas where crime is likely to occur. However, they often rely on historical arrest data, which can reinforce existing patterns of racial profiling and disproportionate targeting of minority communities.
The Bottom Line
As with so many of the risks associated with AI, the most crucial factor is your judgment.
To quote my earlier article on data privacy risks…
Please remember, it’s purely on you to ensure that anything you share on behalf of the organization is true (see Hallucinations Part 2, concluding soapbox!).
Indeed, the AI we’re using now is the worst AI we’ll ever see, and the riskiest. There’s much promise, much complexity, and much to learn. Start safely and proceed with caution, but do start and do use these tools!
Just as a note, I still am amazed by the incredible realism of these text-to-image generation tools. Except when I’m frustrated by the misspellings, extra fingers, and complete misses on simple prompts. But even the worst AI-generated image is better than I could do and often better than what I could find in image searches.
I’ll admit to feeling guilty about each image generation after reading this article from MIT about the environmental impact of AI.
This non-academic experiment was published widely, including in this NPR article.
The surgeon is his mother. But our culture strongly associates “surgeon” with a male, which causes lots of folks (half or more in some studies) to be unable to answer. So don’t feel bad if you couldn’t figure it out! This McSweeney’s article lists quite a few more similar ones. Very enlightening to read and ponder.