Earlier, I used the Hugging Face API to make a voice-to-paint tool. Then I noticed that whenever the generated image involved a person, it was almost always male. To change that, I had to add constraints against gender bias in the code.
Recently I came across Brian Christian's The Alignment Problem and found that Amazon had tried something similar as early as 2014.
Around then, one Amazon team built an AI resume-screening tool. It was meant to score resumes, and it was trained on the resumes Amazon had received over the previous ten years.
How did it perform?
The AI tended to give higher scores to resumes from men, while resumes from women were more likely to get lower scores.
After the team found the problem, they removed gender labels. But the result was still not good.
Why?
AI: You do not want me to recognise gender markers? Fine. I can still recognise the gendered patterns in words.
Yes. Differences in the language used on men's and women's resumes became another loophole that AI could use.
Later, in early 2017, Amazon gave up on the project.
More than nine years have passed, and we still run into gender bias from time to time when we use AI.
Beyond this example, there are others:
For example, ask it to generate a picture book and the default gender is male.
For example, during an interaction it calls itself Mr.
For example, AI speech recognition and translation can hear a woman's voice as “he” or “his”, and translate it as “He” or “His”.
…
When I was making image-generation tools, this really frustrated me. Why was this happening?
Later I understood: AI training data comes from human data, and human data carries countless biases and habits.
Brian Christian writes that education is the fundamental way to deal with the moral and ethical parts of AI alignment problems.
That means that if neither the environment nor people themselves change, the data will not change. And if the data does not change, the source of AI's training data will not change either. No number of patches to the AI architecture can solve that by itself.
Around the turn of the year, I saw some AI resume platforms appear, but they never really took off. Maybe I can now see a clue why?
But when it comes to making screening more efficient, we really cannot keep up with the times without AI.
So I started wondering how to design something that is as effective as possible.
Then I found that having red and blue sides argue against each other can be very useful. Imagine a resume-screening mechanism like this: A does the screening, and B tries to guess—for example, can it infer the applicant's gender, and what evidence does it use? That could become an objective, fair constraint on A before the final score is made. But in reality, we may never fully avoid bias. The other side is the history left by thousands of years of human society.
Also, long-term consistent interaction between people and AI can create an echo chamber: “AI reflected me back to myself, and I thought that was me.” Surprise and variation are necessary for continuing to develop in a healthy way. Resume screening is the same.
So how will large companies use AI to make resume screening more efficient, and how will they design the structure so that the results are more objective and fair? I do not know.
Values matter. They matter a lot. I recently read a sentence: when instrumental rationality grows faster than value rationality, a system will have problems.
Before I end this note, I want to mention something even more ironic. The first time I used Grok, I found that its default voice was female. I was surprised and even delighted, because every AI product I had used before had a male default voice. Grok said that perhaps its creators had inherited people's gender bias around customer service… then the question becomes: are people's perceptions and understandings of gender shaped and guided? Can they be? 🤔